\\\\im\w\\\\\M\m 

© Publication number: 0 502 539 A2 



© EUROPEAN PATENT APPLICATION 

© Application number: 92103872.5 © Int. CI. 5 : H04N '9/04 



@ Date of filing: 06.03.92 



© Priority: 07.03.91 JP 41639/91 


© 


Applicant: CANON KABUSHIKI KAISHA 


07.03.91 JP 41640/91 




30-2, 3-chome, Shimomaruko, Ohta-ku 


07.03.91 JP 41641/91 




Tokyo(JP) 


01.05.91 JP 100032/91 






01.05.91 JP 100033/91 


© 


Inventor: Shiraishi, Akihiko, c/o Canon 


01.05.91 JB 100034/91 




Kabushiki Kaisha 






30-2, 3-chome, Shimomaruko 


© Date of publication of application: 




Ohta-ku, Tokyo 146(JP) 


09.09.92 Bulletin 92/37 






© Designated Contracting States: 


© 


Representative: Tiedtke, Harro, Dipl.-lng. 


DE FR GB 




Patentanwalte Tiedtke-Buhling- Kinne & 






Partner Bavariaring 4 POB 20 24 03 






W-8000 Munchen 2(DE) 



© Color image pickup apparatus. 



causing only a first color signal which is output from 
a pixel belonging to the same column as that of the 
third color signal to coincide therewith, and a second 
color signal forming circuit for forming the color 
information in accordance with a difference signal 
between a second color signal and a coincidence 
signal obtained by causing only a first color signal 
which is output from a pixel belonging to the same 
row as that of the second color signal to coincide 
therewith and a difference signal between a third 
color signal and a coincidence signal obtained by 
causing only a first color signal which is output from 
a pixel belonging to the same row as that of the third 
color signal to coincide therewith, the first and sec- 
ond color signal forming circuits operating on the 
basis of the first, second, and third color signals 
output from pixels corresponding to the first, second, 
and third color filters, and the color information for- 
ming means switching between the first and second 
color signal forming circuits in accordance with a 
frequency component in a scanning direction of the 
object image or a direction perpendicular to the 
scanning direction. 
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© A color image pickup apparatus for converting 
" an object image into an electrical signal having lu- 

j minance ^information and color information includes 
S an image pickup element arranged in a rectan- 
f gular matrix having pixels arranged at a horizontal 

\ pitch P H and a vertical pitch P v ; 

/ a color filter array having first color filters ar- 
ranged in correspondence with the pixels and having 
an offset sampling structure having a horizontal pitch 
/ 2P H and a vertical pitch P v and offset by P H in a 
^ I horizontal direction, and second and third color filters 
I each having a rectangular matrix sampling structure 
\ having a horizontal pitch 2P H and a vertical pitch 
\ 2P V ; and 

a color information forming circuit having a first 
color signal forming circuit for forming the color 
information in accordance with a difference signal 
CO between a second color signal and a coincidence 
ID signal obtained by causing only a first color signal 
which is output from a pixel belonging to the same 
O column as that of the second color signal to coincide 
W therewith and a difference signal between a third 
q color signal and a coincidence signal obtained by 
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BACKGROUND OF THE INVENTION 
Field of the Invention 



The present invention relates to a color image 
pickup apparatus having an image pickup element 
having a plurality of light-receiving elements 
(pixels) two-dim en si on ally arranged in a matrix 
form and, more particularly, to a color image pic- 
kup apparatus capable of outputting an image hav- 
ing a high resolution, reduced moire, and a high 
S/N ratio. 

Related Background Art 

Figs. 1, 2, and 3 are views showing color filter 
arrays of conventional color solid-state image pic- 
kup elements. Fig. 1 shows a stripe filter array in 
which red filters R, green filters G, and blue filters 
B are vertically arranged to constitute the respec- 
tive stripe filters. Figs. 2 and 3 are views showing 
so-called mosaic filters. Fig. 2 shows a filter array 
in which green filters G are constituted by stripe 
filters, and red and blue filters R and B are verti- 
cally arranged every two filters. The set of two red 
filters and two blue filters is arranged parallel to the 
green filters G constituting each stripe filter. In a 
filter array of Fig. 3, magenta, green, cyan, and 
yellow filters Mg, Gr, Cy, and Ye are arranged in an 
illustrated order to constitute a unit array consisting 
of eight color filters, i.e., two horizontal pixels and 
four vertical pixels. 

The image pickup elements having the above 
color filter arrays pose the following problems. In 
the image pickup element having the color filters 
having the array shown in Fig. 1, a color difference 
signal carrier is generated at a frequency 1/3 of the 
sampling frequency. For this reason, an image 
pickup operation cannot be performed up to a 
frequency 1/2 of the Nyquist frequency serving as 
the sampling frequency to decrease an image res- 
olution. 

In the image pickup element having the array 
shown in Fig. 2, since the R and B filters having 
different bands are arranged in the vertical direc- 
tion, a color moire pattern tends to be formed in 
the vertical direction. In particular, image quality 
deteriorates in chromatic images. 

Since the image pickup element having the 
color filters constituting the array shown in Fig. 3 is 
constituted by complementary color filters having 
wide bands, a color moire tends not to be formed 
as compared with the image pickup element having 
the color filters constituting the array shown in Fig. 
2. However, color difference signals have low S/N 
ratios. When an output signal is to be quantized to 
perform digital processing, the color difference sig- 
nals have large quantization errors, resulting in 
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inconvenience. 

In both the image pickup elements having the 
color filters constituting the arrays shown in Figs. 2 
and 3, color difference signal carriers are gen- 
5 erated at a frequency 1/2 of the sampling fre- 
quency. For this reason, data up to a frequency 1/2 
of the Nyquist frequency as the sampling frequen- 
cy cannot be picked up. 

To the contrary, there is provided an image 

io pickup element having a color filter array called a 
Beyer array, as disclosed in U.S. Patent No. 
3,971,065. As shown in Figs. 4A and 4B, assume 
that the horizontal pitch of pixels in the image 
pickup element is defined as P H and the vertical 

is pitch of pixels in the image pickup element is 
defined as P v . In this case, green filters G (Fig. 4A) 
or luminance signal filters Y (Fig. 4B) are arranged 
to have an offset sampling structure having a hori- 
zontal pitch 2P H and the vertical pitch P v and offset 

20 by P H in the horizontal direction. Red and blue 
filters R and B are arranged to have a rectangular 
matrix sampling structure having a horizontal pitch 
2P H and a vertical pitch 2P V . When the image 
pickup element having this Beyer array is used, a 

25 good image having less moire and a high S/N ratio 
is obtained, as is well known. 

The following problem is still posed even if the 
image pickup elements having the Beyer arrays 
are used. Figs. 5A and 5B are graphs showing the 

30 first quadrants obtained when the positions of sig- 
nal carriers generated by the color filter image 
pickup elements shown in Figs. 4A and 4B are 
plotted on the two-dimensional frequency plane 
(f H ,fv)- In the color filter image pickup element 

35 shown in Fig. 4A, an output signal from each pixel 
is directly switched to form a luminance signal. In 
the color filter image pickup element shown in Fig. 
4B, only signals from pixels corresponding to the Y 
filters are used to form a luminance signal. 

40 In either case, color difference signal carriers 

are apparently generated at (1/2P H ,0) and (0,1/2P V ) 
in the frequency space. That is, even in the image 
pickup element having the Beyer array, the color 
difference signal carrier is generated at a frequen- 

45 cy 1/2 of the sampling frequency. Therefore, data 
cannot be picked up to a frequency 1/2 of the 
Nyquist frequency serving as the sampling fre- 
quency. 

In a simple synthetic luminance signal obtained 
so by switching output signals from the pixels or using 
signals from the pixels corresponding to G filters, a 
luminance signal having accurate spectral char- 
acteristics cannot be obtained. Therefore, the resul- 
tant luminance signal adversely affects color repro- 
55 ducibility or the like of an output image. For this 
reason, processing for substituting only a low-fre- 
quency component of the luminance signal with a 
luminance signal having accurate spectral char- 
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acteristics has been conventionally performed. 
However, a circuit for forming the luminance signal 
having the accurate spectral characteristics be- 
comes large in size. 

SUMMARY OF THE INVENTION 

The present invention has been made in con- 
sideration of the above situation, and has as its 
object to provide a color image pickup apparatus 
capable of obtaining a luminance signal whose 
spectral characteristics are corrected by a simple 
technique, and generating a good image having a 
high resolution, reduced moire, and a high S/N 
ratio. 

In order to achieve the above object according 
to the first aspect of the present invention, there is 
provided a color image pickup apparatus for con- 
verting an object image into an electrical signal 
having luminance information and color information, 
comprising: 

a. an image pickup element arranged in a rec- 
tangular matrix having pixels arranged at a hori- 
zontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels and 
having an offset sampling structure having a 
horizontal pitch 2P H and a vertical pitch P v and 
offset by P H in a horizontal direction, and sec- 
ond and third color filters each having a rectan- 
gular matrix sampling structure having a hori- 
zontal pitch 2P H and a vertical pitch 2P V ; and 

c. color information forming means having first 
color signal forming means for forming the color 
information in accordance with a difference sig- 
nal between a second color signal and a co- 
incidence signal obtained by causing only a first 
color signal which is output from a pixel belong- 
ing to the same column as that of the second 
color signal to coincide therewith and a dif- 
ference signal between a third color signal and a 
coincidence signal obtained by causing only a 
first color signal which is output from a pixel 
belonging to the same column as that of the 
third color signal to coincide therewith, and sec- 
ond color signal forming means for forming the 
color information in accordance with a difference 
signal between a second color signal and a 
coincidence signal obtained by causing only a 
first color signal which is output from a pixel 
belonging to the same row as that of the second 
color signal to coincide therewith and a dif- 
ference signal between a third color signal and a 
coincidence signal obtained by causing only a 
first color signal which is output from a pixel 
belonging to the same row as that of the third 
color signal to coincide therewith, the first and 
second color signal forming means operating on 



the basis of the first, second, and third color 
signals output from pixels corresponding to the 
first, second, and third color filters, and the color 
information forming means switching between 
s the first and second color signal forming means 

in accordance with a frequency component in a 
scanning direction of the object image or a 
direction perpendicular to the scanning direction. 
According to the arrangement of the first as- 
w pect, the color difference signal carriers of a mon- 
ochrome object at positions (1/2P H ,0) and (0,1/2P V ) 
in the two-dimensional frequency space are can- 
celed to suppress the moire. 

In order to achieve the above object according 
75 to the second aspect of the present invention, there 
is provided a color image pickup apparatus for 
converting an object image into an electrical signal 
having luminance information and color information, 
comprising: 

20 a. an image pickup element arranged in a rec- 
tangular matrix having pixels arranged at a hori- 
zontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels and 

25 having an offset sampling structure having a 
0 \*^ horizontal pitch 2P H and a vertical pitch P v and 
J^^U)*? offset by P H in a horizontal direction, and sec- 
J^ond and third color filters each having a rectan- 
■J c £» gular matrix sampling structure having a hori- 
30 zontal pitch 2P H and a vertical pitch 2P V ; and 

c. color information forming means for forming 
the color information in accordance with a dif- 
ference signal between a second color signal 
and a coincidence signal obtained by causing 

35 only a first color signal which is output from a 
pixel belonging to the same column or row as 
that of the second color signal to coincide there- 
with and a difference signal between a third 
color signal and a coincidence signal obtained 

40 by causing only a first color signal which is 
output from a pixel belonging to the same col- 
umn or row as that of the third color signal to 
coincide therewith on the basis of the first, sec- 
ond, and third color signals output from pixels 

45 corresponding to the first, second, and third 
color filters. 

According to the arrangement of the second 
aspect, the color difference signal carriers of a 
monochrome object at positions (1/2P H ,0) and 
so (0,1 /2P V ) in the two-dimensional frequency space 
are canceled. 

In order to achieve the above object according 
to still another aspect of the present invention, 
there is provided a color image pickup apparatus 
55 for converting an object image into an electrical 
signal having luminance information and color in- 
formation, comprising: 

a. an image pickup element arranged in a rec- 
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tangular matrix having pixels arranged at a hori- 
zontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels and 
having an offset sampling structure having a 
horizontal pitch 2P H and a vertical pitch P v and 
offset by P H in a horizontal direction, and sec- 
ond and third color filters each having a rectan- 
gular matrix sampling structure having a hori- 
zontal pitch 2P H and a vertical pitch 2P V ; 

c. an optical low-pass filter having an optical 
member for splitting an incident beam into two 
beams spaced apart from each other by a dis- 
tance D in a direction forming an angle 6 with 
respect to a scanning direction of the image 
pickup element in clockwise and counterclock- 
wise directions, the optical low-pass filter being 
arranged in an image pickup optical system and 
satisfying the following condition: 

O.8P H Pv/|PHSin0 + P v cosfi| £ D ^ 
1 .2P H P v /|pHSin0 + P v cos0| 
for 0 S 9 £ tt/2; and 



d. color information forming means for forming 
the color information in accordance with a dif- 
ference signal between a second color signal 
and a coincidence signal obtained by causing 
only a first color signal which is output from a 
pixel belonging to the same column or row as 
that of the second color signal to coincide there- 
with and a difference signal between a third 
color signal and a coincidence signal obtained 
by causing only a first color signal which is 
output from a pixel belonging to the same col- 
umn or row as that of the third color signal to 
coincide therewith on the basis of the first, sec- 
ond, and third color signals output from pixels 
corresponding to the first, second, and third 
color filters. 

In order to achieve the above object according 
to the fourth aspect of the present invention, there 
is provided a color image pickup apparatus for 
converting an object image into an electrical signal 
having luminance information and color information, 
comprising: 

an image pickup element arranged in a rectan- 
gular matrix having pixels arranged at a horizontal 
pitch P H and a vertical pitch P v ; 

a color filter array having first color filters ar- 
ranged in correspondence with the pixels and hav- 
ing an offset sampling structure having a horizontal 
pitch 2P H and a vertical pitch P v and offset by P H 
in a horizontal direction, and second and third color 
filters each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and a verti- 
cal pitch 2P V ; 



SNSDOC1D: <£P_06O2639A2JLj> 



an optical low-pass filter having an optical 
member for splitting an incident beam into two 
beams spaced apart from each other by a distance 
D in a direction forming an angle 6 with respect to 
s a scanning direction of the image pickup element 
in clockwise and counterclockwise directions, the 
optical low-pass filter being arranged in an image 
pickup optical system and satisfying the following 
condition: 

10 

0.8P H P v /|PHSine + P v cos0| S D £ 
1 .2P H Pv|PHSin0 + PvCose| 
for 0 £ 0 ^ tt/2; and 

;s color information forming means having first 

color signal forming means for forming the color 
information in accordance with a difference signal 
between a second color signal and a coincidence 
signal obtained by causing only a first color signal 

20 which is output from a pixel belonging to the same 
column as that of the second color signal to co- 
incide therewith and a difference signal between a 
third color signal and a coincidence signal obtained 
by causing only a first color signal which is output 

25 from a pixel belonging to the same column as that 
of the third color signal to coincide therewith, and 
second color signal forming means for forming the 
color information in accordance with a difference 
signal between a second color signal and a co- 

30 incidence signal obtained by causing only a first 
color signal which is output from a pixel belonging 
to the same row as that of the second color signal 
to coincide therewith and a difference signal be- 
tween a third color signal and a coincidence signal 

35 obtained by causing only a first color signal which 
is output from a pixel belonging to the same row as 
that of the third color signal to coincide therewith, 
the first and second color signal forming means 
operating on the basis of the first, second, and 

40 third color signals output from pixels corresponding 
to the first, second, and third color filters, and the 
color information forming means switching between 
the first and second color signal forming means in 
accordance with a frequency component in a scan- 

45 ning direction of the object image or a direction 
perpendicular to. the scanning direction. 

In the arrangement of the third aspect, the 
color difference signal carrier of the monochrome 
object at the position (1/2P H ,0) or (0,1/2P V ) in the 

50 two-dimensional frequency space is canceled, and 
the color difference signal carriers at positions 
(i1Z2PH.il/2Pv) are suppressed. In the arrange- 
ment of the fourth aspect, the color difference 
signal carriers of the monochrome object at the 

55 positions (1/2P H ,0) and (0,1/2P V ) in the two-dimen- 
sional frequency space are canceled, and the color 
difference signal carriers at the positions 
(i1Z2P H ,i1/2P v ) are suppressed. 

5 
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In order to achieve the above object according 
to the fifth aspect of the present invention, there is 
provided a color image pickup apparatus for con- 
verting an object image into an electrical signal 
having luminance information and color information, 
comprising: 

a. an image pickup element arranged in a rec- 
tangular matrix having pixels arranged at a hori- 
zontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels and 
having an offset sampling structure having a 
horizontal pitch 2P H and a vertical pitch P v and 
offset by Ph in a horizontal direction, and sec- 
ond and third color filters each having a rectan- 
gular matrix sampling structure having a hori- 
zontal pitch 2P H and a vertical pitch 2P V ; 

c. an optical low-pass filter having an optical 
member for splitting an incident beam into two 
beams spaced apart from each other by a dis- 
tance D in a direction forming an angle 6 with 
respect to a scanning direction of the image 
pickup element in clockwise and counterclock- 
wise directions, the optical low-pass filter being 
arranged in an image pickup optical system and 
satisfying the following condition: 

O.8P H P v /|PHSin0 +P v cos0| 3 D i 
1 .2P H P v /|PHSin0 + P v cos0| 
for 0 ^ 6 £ tt/2; 



d. color information forming means for forming 
the color information in accordance with a first 
difference signal between a second color signal 
and a coincidence signal obtained by causing 
only a first color signal which is output from a 
pixel belonging to the same column or row as 
that of the second color signal to coincide there- 
with and a second difference signal between a 
third color signal and a coincidence signal ob- 
tained by causing only a first color signal which 
is output from a pixel belonging to the same 
column or row as that of the third color signal to 
coincide therewith on the basis of the first, sec- 
ond, and third color signals output from pixels 
corresponding to the first, second, and third 
color filters; and 

e. luminance information forming means for for- 
ming the luminance information by multiplying 
the first and second difference signals with con- 
stants, respectively, and adding the multiplied 
signals to a signal obtained by synthesizing at 
least one of the first, second, and third color 
signals. 

In order to achieve the above object according 
to the sixth aspect of the present invention, there is 
provided a color image pickup apparatus for con- 
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verting an object image into an electrical signal 
having luminance information and color information, 
comprising: 

a. an image pickup element arranged in a rec- 
5 tangular matrix having pixels arranged at a hori- 
zontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels and 
having an offset sampling structure having a 

w horizontal pitch 2P H and a vertical pitch P v and 
offset by P H in a horizontal direction, and sec- 
ond and third color filters each having a rectan- 
gular matrix sampling structure having a hori- 
zontal pitch 2P H and a vertical pitch 2P V ; 

15 c. an optical low-pass filter having an optical 
member for splitting an incident beam into two 
beams spaced apart from each other by a dis- 
tance D in a direction forming an angle d with 
respect to a scanning direction of the image 

20 pickup element in clockwise and counterclock- 
wise directions, the optical low-pass filter being 
arranged in an image pickup optical system and 
satisfying the following condition: 

25 O.8P H P v /|PHSin0 + P v cose| ^ D ^ 
1 .2P H Pv/|PHSin0 + P v cos0| 
for 0 ^ 6 £ tt/2; 



30 d. color information forming means having first 
color signal forming means for forming the color 
information in accordance with a first difference 
signal between a second color signal and a 
coincidence signal obtained by causing only a 

35 first color signal which is output from a pixel 

belonging to the same column as that of the 
second color signal to coincide therewith and a 
second difference signal between a third color 
signal and a coincidence signal obtained by 

40 causing only a first color signal which is output 
from a pixel belonging to the same column as 
that of the third color signal to coincide there- 
with, and second color signal forming means for 
forming the color information in accordance with 

45 a third difference signal between a second color 
signal and a coincidence signal obtained by 
causing only a first color signal which is output 
from a pixel belonging to the same row as that 
of the second color signal to coincide therewith 

so and a fourth difference signal between a third 
color signal and a coincidence signal obtained 
by causing only a first color signal which is 
output from a pixel belonging to the same row 
as that of the third color signal to coincide 

55 therewith, the first and second color signal for- 
ming means operating on the basis of the first, 
second, and third color signals output from pix- 
els corresponding to the first, second, and third 

6 
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color filters, and the color information forming 
means switching between the first and second 
color signal forming means in accordance with a 
frequency component in a scanning direction of 
the object image or a direction perpendicular to 
the scanning direction; and 

e. luminance information forming means for for- 
ming the luminance information by multiplying 
the first or third difference signal and the second 
or fourth difference signal with constants, re- 
spectively, and adding the multiplied signals to 
a signal obtained by synthesizing at least one of 
the first, second, and third color signals. 
In the arrangement of the fifth aspect, the color 
difference signal carrier of the monochrome object 
at the position (1/2P H ,0) or (0,1 /2P V ) in the two- 
dimensional frequency space is canceled, and the 
color difference signal carriers at positions 
(±1/2P H ,±1/2P V ) are suppressed. In the arrange- 
ment of the sixth aspect, the color difference signal 
carriers of the monochrome object at the positions 
(1/2P Hl 0) and (0,1 /2P V ) in the two-dimensional fre- 
quency space are canceled, and the color dif- 
ference signal carriers at the positions 
(±1/2P H ,±1/2P V ) are suppressed. 

In addition, in the arrangements of the fifth and 
sixth aspects, the signals obtained by multiplying 
the difference signals with the constants are added 
to the synthesized luminance information to correct 
spectral characteristics of the luminance informa- 
tion. 

In order to achieve the above object according 
to the seventh aspect of the present invention, 
there is provided a color image pickup apparatus 
for converting an object image into an electrical 
signal having luminance information and color in- 
formation, comprising: 

a. an image pickup element arranged in a rec- 
tangular matrix having pixels arranged at a hori- 
zontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels and 
having an offset sampling structure having a 
horizontal pitch 2P H and a vertical pitch P v and 
offset by P H in a horizontal direction, and sec- 
ond and third color filters each having a rectan- 
gular matrix sampling structure having a hori- 
zontal pitch 2P H and a vertical pitch 2P V ; 

c. color information forming means for forming 
the color information in accordance with a first 
difference signal between a second color signal 
and a coincidence signal obtained by causing 
only a first color signal which is output from a 
pixel belonging to the same column or row as 
that of the second color Signal to coincide there- 
with and a second difference signal between a 
third color signal and a coincidence signal ob- 
tained by causing only a first color signal which 
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is output from a pixel belonging to the same 
column or row as that of the third color signal to 
coincide therewith on the basis of the first, sec- 
ond, and third color signals output from pixels 
5 corresponding to the first, second, and third 

color filters; and 

d. luminance information forming means for for- 
ming the luminance information by multiplying 
the first and second difference signals with con- 
io stants, respectively, and adding the multiplied 
signals to a signal obtained by synthesizing at 
least one of the first, second, and third color 
signals. 

In the arrangement of the seventh aspect, the 
75 color difference signal carrier of the monochrome 
object at the position (1/2P H ,0) or (0,1/2P V ) in the 
two-dimensional frequency space is canceled. In 
addition, in the arrangements of the seventh as- 
pect, the signals obtained by multiplying the first 
20 and second difference signals with the constants 
are added to the synthesized luminance informa- 
tion to correct spectral characteristics of the lu- 
minance information. 

In order to achieve the above object according 
25 to the eighth aspect of the present invention, there 
is provided a color image pickup apparatus for 
converting an object image into an electrical signal 
having luminance information and color information, 
comprising: 

30 a. an image pickup element arranged in a rec- 
tangular matrix having pixels arranged at a hori- 
zontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color fitters 
arranged in correspondence with the pixels and 

35 having an offset sampling structure having a 
horizontal pitch 2P H and a vertical pitch P v and 
offset by P H in a horizontal direction, and sec- 
ond and third color filters each having a rectan- 
gular matrix sampling structure having a hori- 

40 zontal pitch 2P H and a vertical pitch 2P V ; 

c. color information forming means having first 
color signal forming means for forming the color 
information in accordance with a first difference 
signal between a second color signal and a 

45 coincidence signal obtained by causing only a 
first color signal which is output from a pixel 
belonging to the same column as that of the 
second color signal to coincide therewith and a 
second difference signal between a third color 

so signal and a coincidence signal obtained by 
causing only a first color signal which is output 
from a pixel belonging to the same column as 
that of the third color signal to coincide there- 
with, and second color signal forming means for 

55 forming the color information in accordance with 
a third difference signal between a second color 
. signal and a coincidence signal obtained by 
causing only a first color signal which is output 
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from a pixel belonging to the same row as that 
of the second color signal to coincide therewith 
and a fourth difference signal between a third 
color signal and a coincidence signal obtained 
by causing only a first color signal which is 
output from a pixel belonging to the same row 
as that of the third color signal to coincide 
therewith, the first and second color signal for- 
ming means operating on the basis of the first, 
second, and third color signals output from pix- 
els corresponding to the first, second, and third 
color filters, and the color information forming 
means switching between the first and second 
color signal forming means in accordance with a 
frequency component in a scanning direction of 
the object image or a direction perpendicular to 
the scanning direction; and 
d. luminance information forming means for for- 
ming the luminance information by multiplying 
the first or third difference signal and the second 
or fourth difference signal with constants, re- 
spectively, and adding the multiplied signals to 
a signal obtained by synthesizing at least one of 
the first, second, and third color signals. 
In the arrangement of the eighth aspect, the 
color difference signal carriers of the monochrome 
object at the positions (1/2P H ,0) and (0,1/2P V ) in the 
two-dimensional frequency space are canceled, 
and the moire can be suppressed. 

In addition, in the arrangement of the eighth 
aspect, the signals obtained by multiplying the first 
and second or third and fourth difference signals 
with the constants are added to the synthesized 
luminance information to correct spectral character- 
istics of the luminance information. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a view showing a conventional color 
filter array; 

Fig. 2 is a view showing another conventional 
color filter array; 

Fig. 3 is a view showing still another conven- 
tional color filter array; 

Figs. 4A and 4B is are views showing conven- 
tional color filter arrays having Beyer arrays, 
respectively; 

Figs. 5A and 5B are graphs showing the posi- 
tions of signal carriers of the color filter arrays 
shown in Figs. 4A and 4B, respectively; 
Fig. 6 is a block diagram showing the first 
embodiment of the present invention; 
Figs. 7A and 7B are views for explaining the first 
embodiment; 

Figs. 8A and 8B are block diagrams showing 

arrangements of a decision circuit; 

Fig. 9 is a block diagram showing the second 

embodiment; 
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Figs. 10A and 10B are views for explaining the 
second embodiment; 

Fig. 11 is a block diagram showing the third 
embodiment of the present invention; 
5 Fig. 12 is a view for explaining signal processing 

of the third embodiment; 

Fig. 13 is a view for explaining signal processing 
of the fourth embodiment of the present inven- 
tion; 

w Fig. 14 is a block diagram showing the fifth 
embodiment of the present invention; 
Fig. 15 is a view for explaining signal processing 
of the fifth embodiment; 

Fig. 16 is a view for explaining signal processing 
is of the sixth embodiment of the present inven- 
tion; 

Fig. 17 is a block diagram showing the seventh 

embodiment of the present invention; 

Fig. 18 is a view for explaining the seventh 

20 embodiment; 

Fig. 19 is a view showing an optical low-pass 
filter used in the seventh embodiment; 
Fig. 20 is a view for explaining the optical low- 
pass filter shown in Fig. 19; 

25 Fig. 21 is a view of an optical low-pass filter 
used in a modification of the seventh embodi- 
ment; 

Fig. 22 is a view of an optical low-pass filter 
used in another modification of the seventh em- 

30 bodiment; 

Fig. 23 is a view for explaining the optical low- 
pass filter shown in Fig. 22; 
Fig. 24 is a view for explaining the eighth em- 
bodiment of the present invention; 

35 Fig. 25 is a view for explaining the optical low- 

pass filters shown in Figs. 19 and 21; 
Fig. 26 is a view of an optical low-pass filter 
used in a modification of the eighth embodi- 
ment; 

40 Fig. 27 is a view for explaining the optical low- 
pass filter shown in Fig. 26; 
Fig. 28 is a block diagram showing the ninth 
embodiment of the present invention; 
Fig. 29 is a view for explaining the ninth em- 

45 bodiment; 

Fig. 30 is a view for explaining the tenth em- 
bodiment of the present invention; 
Fig. 31 is a view for explaining the eleventh 
embodiment of the present invention; 

so Fig. 32 is a block diagram showing the twelfth 
embodiment of the present invention; 
Fig. 33 is a block diagram showing an arrange- 
ment of a decision circuit; 

Fig. 34 is a block diagram showing another 
55 arrangement of the decision circuit; 

Fig. 35 is a block diagram showing the thirteenth 

embodiment of the present invention; 

Fig. 36 is a view for explaining the thirteenth 
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embodiment; 

Fig. 37 is a view showing an optical low-pass 
filter used in a modification of the thirteenth 
embodiment; 

Fig. 38 is a view for explaining the optical low- 
pass filter shown in Fig. 37; 
Fig. 39 is a view of an optical low-pass filter 
used in a modification of the thirteenth embodi- 
ment; 

Fig. 40 is a view showing an optical low-pass 
filter used in another modification of the thir- 
teenth embodiment; 

Fig. 41 is a view of the optical low-pass filter 
shown in Fig. 40; 

Fig. 42 is a view for explaining the fourteenth 
embodiment of the present invention; 
Fig. 43 is a view for explaining the optical low- 
pass filters shown in Figs. 37 and 39; 
Fig. 44 is a view of an optical low-pass filter 
used in a modification of the fourteenth embodi- 
ment; 

Fig. 45 is a view for explaining the optical low- 
pass filter shown in Fig. 44; 
Fig. 46 is a block diagram showing the fifteenth 
embodiment of the present invention; 
Fig. 47 is a view showing a Beyer array; 
Fig. 48 is a view showing another Beyer array; 
Fig. 49 is a block diagram showing the sixteenth 
embodiment of the present invention; 
Fig. 50 is a block diagram showing the seven- 
teenth embodiment of the present invention; 
:Fig. 51 is a block diagram showing the eigh- 
teenth and nineteenth embodiments of the 
present invention; 

Fig. 52 is a view for explaining signal processing 
of the eighteenth and twentieth embodiments; 
Fig. 53 is a view for explaining signal processing 
of the nineteenth and twenty-first embodiments; 
Fig. 54 is a block diagram showing the twentieth 
and twenty-first embodiments; 
Fig. 55 is a block diagram showing the twenty- 
second embodiment of the present invention; 
Figs. 56A and 56B are views for explaining the 
twenty-second embodiment; and 
Fig. 57 is a block diagram showing the twenty- 
third embodiment of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

The present invention will be described in de- 
tail with reference to the preferred embodiments 
hereinafter. 

Fig. 6 is a block diagram showing a "color 
image pickup apparatus" as the first embodiment 
of the present invention. R, G, and B filters (filter 
array) having a Beyer array shown in Fig. 4A are 
arranged in an image pickup element (sensor) 101. 
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An image signal read from the image pickup ele- 
ment 101 pixel by pixel is separated into R t G. and 
B signals by a color separation unit 102. The R, G, 
and B signals are subjected to white balance ad- 
5 justment by a white balance unit 1 1 1 on the basis 
of color temperature information obtained from a 
white balance sensor (AWB) 120. An output from 
the white balance unit 1 1 1 is subjected to y correc- 
tion by a 7 correction unit 112. An output from the 

w y correction unit 112 is A/D-converted by an A/D 
(analog-to-digital) converter 103. 

A luminance signal is switched by a switch 
circuit (SWY) 126 and is rearranged in a read 
order. The rearranged luminance signal is extracted 

15 as a high-frequency component Y H of the lumi- 
nance signal by a band-pass filter (BPF) 116. The 
high-frequency component Y H of the luminance 
signal is added by an adder 117 to a low-frequency 
component Y L of the luminance signal obtained by 

20 a technique to be described later. A sum signal is 
converted by a D/A (digital-to-analog) converter 
118, and an analog signal is output from the D/A 
converter 118. 

Meanwhile, a G (>th power) signal of outputs 

25 from the A/D converter 103 is separated by a 
switch (SW) 128 into Gi ( 7 th power) and G 2 (7th 
power) signals located at the illustrated positions at 
two timings, i.e., timing 1 shown in Fig. 7A and 
timing 2 shown in Fig. 7B. This operation can be 

30 performed by operating the switch 128, e.g., every 
horizontal scanning period. Switching between tim- 
ings 1 and 2 is performed by a decision circuit 131 
(to be described later) in accordance with a lu- 
minance signal of an object to be photographed. 

35 The separated Gi ( 7 th power) and G2 (7th power) 
signals are input together with an R ( 7 th power) 
signal and a B (7th power) signal to interpolation 
filters 106, 107, 108, and 109, respectively, thereby 
obtaining coincident R ( 7 th power), G) (7th power), 

40 G2 (7th power), and B (7th power) signals. In 
addition to the coincidence operation by interpola- 
tion in the interpolation filters 106 to 109, linear 
processing such as two-dimensional low-pass filter- 
ing and edge emphasis are performed. Since these 

45 processing operations are linear processing oper- 
ations, they can be performed after addition and 
matrix processing (both will be described in detail 
later). 

The coincident R (7th power) and Gi (7th 
50 power) signals are subtracted from each other by 
an adder 129, and an (R ( 7 th power) - Gi (7th 
power)) signal is output from the adder 129. Simi- 
larly, the B ( 7 th power) and G2 (7th power) signals 
are subtracted from each other by an adder 130, 
55 arid a (B ( 7 th power) - G2 (7th power)) signal is 
output from the adder 130. These output signals 
are input to a color difference matrix processor 
113, and the following matrix operation is per- 
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formed to derive color difference signals R-Y and 
B-Y. 



R-Y 



0 .70-0 . 11 
-0 .304-0 . 89 



Assume that switching between the Gi (7th 10 
power) and G2 {7th power) signals is performed at 
timing 1 shown in Fig. 7A, and that a monochrome 
object at a position (1/2P H ,0) in the frequency 
space is picked up by the image pickup element 
101. This object represents vertical fringes having 15 
a period 2P H . For this object, conditions R (7th 
power) = G1 (7th power) and B (7th power) = G2 
(7th power) are established. The color difference 
signals, i.e., the (R (7th power) - G1 (7th power)) 
signal and the (B ( 7 th power) - G2 (7th power)) 20 
signal respectively output from the adders 129 and 
130 become zero. Therefore, the color difference 
signals R-Y and B-Y output from the color dif- 
ference matrix processor 1 1 3 become zero and are 
not output. This indicates that the carrier of the 25 
color difference signal at the frequency point 
(1/2P H ,0) is canceled. In other words, the carrier of 
the R (7th power) signal is set in phase with that of 
the G1 (7th power), and the carrier of the B (7th 
power) is also in phase with that of the G 2 (7th 30 
power) signal at the frequency point (1/2P H ,0). 
Since these difference signals, i.e., the (R (7th 
power) - G1 (7th power)) signal and the (B (7th 
power) - G2 (7th power)) signal, can cancel their 
carriers at this frequency, the carriers of the color 35 
difference signals are not generated. 

Assume that switching between the Gi (7th 
power) and G 2 (7th power) signals is performed at 
timing 2 shown in Fig. 7B, and that a monochrome 
object at a position (0,1 /2P V ) in the frequency 40 
space is picked up by the image pickup element 
101. This object represents horizontal fringes hav- 
ing a period 2P V . For this object, the color dif- 
ference signals, i.e., the (R (7th power) - Gi (7th 
power)) signal and the (B ( 7 th power) - G2 (7th 45 
power)) signal respectively output from the adders 
129 and 130 become zero. Therefore, the color 
difference signals R-Y and B-Y output from the 
color difference matrix processor 113 become zero 
and are not output. This indicates that the carrier of so 
the color difference signal at the frequency point 
(0,1/2P V ) is canceled. In other words, the carrier of 
the R (7th power) signal is set in phase with that of 
the Gi (7th power), and the carrier of the B (7th 
power) is also in phase with that of the G 2 (7th ss 
power) signal at the frequency point (0,1/2P V ). 
Since these difference signals, i.e., the (R (7th 
power) - Gi (7th power)) signal and the (B ( 7 th 



power) - G2 (7th power)) signal, can cancel their 
carriers at this frequency,, the carriers of the color 
difference signals are not generated. 

These color difference signals are converted 
into analog signals by D/A converters 1 14 and 1 15, 
and the analog signals are output. A low-frequency 
component of the luminance signal is generated by 
a luminance signal generation circuit 127 in accor- 
dance with outputs from the interpolation filters 
106, 107, 108, and 109 as follows: 

Y L = 0.30R (7th power) + [«Gi (7th power) + 
j9G 2 (7th power)] + 0.1 1B ( 7 th power) 
for a + £ = 0.59 

The low-frequency component is added to the 
high-frequency component Y H by the adder 117. 
An output from the adder 117 is D/A-converted by 
the D/A converter 118, and an analog signal is 
output. Since the color difference signals R-Y and 
B-Y and the low-frequency component Y u of the 
luminance signal generally have bandwidths suffi- 
ciently narrower than the luminance signal Y, oper- 
ations of the adders 129 and 130, the color dif- 
ference matrix processor 113, the luminance signal 
generation circuit 127, and the like for the interpo- 
lated coincident R (7th power), Gi (7th power), G2 
(7th power), and B ( 7 th power) signals may be 
performed using a clock having a lower clock rate 
than that of the luminance signal Y by extraction or 
interlacing. 

When an output signal obtained from the pro- 
cessing block shown in Fig. 6 is to be recorded in 
an analog form, the D/A converters 118, 114, and 
1 1 5 are necessary. However, when this output sig- 
nal is recorded in a digital form in a magnetic 
medium, an optomagnetic medium, an E 2 PROM 
(Electrically Erasable PROM), or the like, the above 
D/A converters can be omitted. 

The decision circuit 131 will be described be- 
low. Fig. 8A shows an arrangement of this decision 
circuit. The luminance signal Y output from the 
switch circuit 126 is filtered by a band-pass filter 
(H-BPF) 31 in the horizontal direction to extract a 
horizontal high-frequency component. This extract- 
ed output is input to a comparator (comp) 32 and is 
compared with a predetermined threshold value. 
When it is decided that the input signal level is 
higher than the threshold level and the horizontal 
high-frequency component value is large, timing 1 
is selected to cancel the carrier of the horizontal 
color difference signal (Fig. 7A). Otherwise, timing 
2 for canceling the vertical color difference signal 
carrier is selected (Fig. 7B). 

The decision circuit 131 may have an arrange- 
ment show in Fig. 8B. That is, the luminance signal 
Y output from the switch circuit 126 is filtered by a 
.band-pass filter (V-BPF) 61 in the vertical direction 
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to extract a vertical high-frequency component. 
This extracted output is input to a comparator 
(comp) 62 and is compared with a predetermined 
threshold value. When it is decided that the input 
signal level is higher than the threshold level and 
the vertical high-frequency component value is 
large, timing 2 is selected to cancel the carrier of 
the vertical color difference signal. Otherwise, tim- 
ing 1 for canceling the horizontal color difference 
signal carrier is selected. 

As described above, since the color filters con- 
stituting the Beyer array are used, the moire is 
minimized, and a high S/N ratio can be obtained. In 
addition, since a signal processing means suitable 
for the Beyer array is used, a higher resolution can 
be obtained. 

The second embodiment of the present inven- 
tion will be described below. 

Fig. 9 is a block diagram showing a signal 
processing section of a color image pickup appara- 
tus of the second embodiment of the present in- 
vention. Y, R, and B filters having a Beyer array 
shown in Fig. 4B are arranged in an image pickup 
element (sensor) 401 . An image signal read from 
the image pickup element 401 pixel by pixel is 
separated into Y, R, and B signals by a color 
separation unit 402. The Y, R, and B signals are 
subjected to white balance adjustment by a white 
balance unit 41 1 on the basis of color temperature 
information obtained from a white balance sensor 
(AWB) 420. An output from the white balance unit 
41 1 is subjected to y correction by a 7 correction 
unit 412. An output from the 7 correction unit 412 
is A/D-converted by an A/D (analog-to-digital) con- 
verter 403. 

A luminance signal is obtained as follows. An 
offset sampling structure of a Y (7th power) signal 
is two-dimensionally interpolated by an interpola- 
tion filter 425, and an output from the interpolation 
filter 425 is D/A-converted by a D/A converter 418, 
An analog signal is thus output from the D/A con- 
verter 418. In addition to the coincidence operation 
by interpolation in the interpolation filter 425, pro- 
cessing such as two-dimensional low-pass filtering 
and edge emphasis are performed. 

Meanwhile, a Y (7th power) signal of outputs 
from the A/D converter 403 is separated by a 
switch 428 into Y1 , (7th power) and Y 2 (7th power) 
signals located at the illustrated positions at two 
timings, i.e., timing 1 shown in Fig. 1 0A and timing 
2 shown in Fig. 10B. This operation can be per- 
formed by operating the switch 428, e.g., every 
horizontal scanning period. Switching between tim- 
ings 1 and 2 is performed by a decision circuit 431 
(to be described later) in accordance with a lu- 
minance signal of an object to be photographed. 

The separated Yi ( 7 th power) and Y 2 (7th 
power) signals are input together with an R (7th 
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power) signal and a B (7th power) signal to inter- 
polation filters 406, 407, 408, and 409, respectively, 
thereby obtaining coincident R (7th power), Yi (7th 
power), Y2 (7th power), and B (7th power) signals. 
5 In addition to the coincidence operation by inter- 
polation in the interpolation filters 406 to 409, linear 
processing such as two-dimensional low-pass filter- 
ing and edge emphasis are performed. Since these 
processing operations are linear processing oper- 

10 ations, they can be performed after addition (to be 
described in detail later). The coincident R (7th 
power) and Yi (7th power) signals are subtracted 
from each other by an adder 429, and an R-Y 
signal is output from the adder 429. Similarly, the B 

75 (7th power) and Y2 (7th power) signals are sub- 
tracted from each other by an adder 430, and a B- 
Y signal is output from the adder 430. 

Assume that switching between the Yi (7th 
power) and Y2 (7th power) signals is performed at 

20 timing 1 shown in Fig. 10A, and that a monoch- 
rome object at a position (1/2P H ,0) in the frequency 
space is picked up by the image pickup element 
401. This object represents vertical fringes having 
a period 2P H . For this object, the signals R-Y and 

25 B-Y respectively output from the adders 429 and 
430 become zero and are not output. This indicates 
that the carrier of the color difference signal at the 
frequency point (1/2P Hf 0) is canceled. In other 
words, the carrier of the R (7th power) signal is set 

30 in phase with that of the Yi (7th power), and the 
carrier of the B (7th power) is also in phase with 
that of the Y 2 (7th power) signal at the frequency 
point (1/2P H ,0). Since these R-Y and B-Y can can- 
cel their carriers at this frequency, the carriers of 

35 the color difference signals are not generated. 

Assume that switching between the Yi ( 7 th 
power) and Y 2 (7th power) signals is performed at 
timing 2 shown in Fig. 10B, and that a monoch- 
rome object at a position (0,1 /2P V ) in the frequency 

40 space is picked up by the image pickup element 
401. This object represents horizontal fringes hav- 
ing a period 2P V . For this object, the color dif- 
ference signals R-Y and B-Y respectively output 
from the adders 429 and 430 become zero. There- 

45 fore, the color difference signals R-Y and B-Y out- 
put from the adders 429 and 430 become zero and 
are not output. This indicates that the carrier of the 
color difference signal at the frequency point 
(0,1 /2P V ) is canceled. In other words, the carrier of 

so the R (7th power) signal is set in phase with that of 
the Yi (7th power), and the carrier of the B ( 7 th 
power) is also in phase with that of the Y 2 (7th 
power) signal at the frequency point (0,1 /2P V ). 
Since these difference signals R-Y and B-Y can 

55 cancel their carriers at this frequency, the carriers 
of the color difference signals are not generated. 
These color difference signals are converted into 
analog signals by D/A converters 414 and 415, and 
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the analog signals are output. Since the color dif- 
ference signals R-Y and B-Y generally have band- 
widths sufficiently narrower than the luminance sig- 
nal Y, operations of the adders 429 and 430, and 
the like for the interpolated coincident R (7th pow- 
er). Yi (7th power), Y 2 (7th power), and B (7th 
power) signals may be performed using a clock 
having a lower clock rate than that of the luminance 
signal Y by extraction or interlacing. 

When an output signal obtained from the pro- 
cessing block shown in Fig. 4A or 4B is to be 
recorded in an analog form, the D/A converters 
418, 414, and 415 are necessary. However, when 
this output signal is recorded in a digital form in a 
magnetic medium, an optomagnetic medium, an 
E 2 PROM (Electrically Erasable PROM), or the like, 
the above D/A converters can be omitted. 

The decision circuit 431 will be described be- 
low. In this embodiment, the decision circuit shown 
in Fig. 8A or 8B is used. In the arrangement of Fig. 
8A, the luminance signal Y output from the inter- 
polation filter 425 is filtered by a band-pass filter 
(H-BPF) 31 in the horizontal direction to extract a 
horizontal high-frequency component. This extract- 
ed output is input to a comparator (comp) 32 and is 
compared with a predetermined threshold value. 
When it is decided that the input signal level is 
higher than the threshold level and the horizontal 
high-frequency component value is large, timing 1 
is selected to cancel the carrier of the horizontal 
color difference signal (Fig. 5A). Otherwise, timing 
2 for canceling the vertical color difference signal 
carrier is selected (Fig. 5B). 

In the arrangement of the decision circuit 
shown in Fig. 8B, the luminance signal Y output 
from the interpolation filter 425 is filtered by a 
band-pass filter (V-BPF) 61 in the vertical direction 
to extract a vertical high-frequency component. 
This extracted output is input to a comparator 
(comp) 62 and is compared with a predetermined 
threshold value. When it is decided that the input 
signal level is higher than the threshold level and 
the vertical high-frequency component value is 
large, timing 2 is selected to cancer the carrier of 
the vertical color difference signal. Otherwise, tim- 
ing 1 for canceling the horizontal color difference 
signal carrier is selected (see Figs. 10A and 10B). 

As described above, since the color filters con- 
stituting the Beyer array are used in the image 
pickup element, appropriate signal processing is 
performed. There is therefore provided a color im- 
age pickup apparatus capable of producing an im- 
age having a high resolution, less moire, and a 
high S/N ratio. 

The present invention will be described in 
more detail with reference to still another embodi- 
ment of the present invention. 

Fig. 11 is a block diagram showing a "color 



image pickup apparatus" as the third embodiment 
of the present invention. R, G, and B filters (filter 
array) having a Beyer array shown in Fig. 4A are 
arranged in an image pickup element (sensor) 101. 
5 An image signal read from the image pickup ele- 
ment 101 pixel by pixel is separated into R, G, and 
B signals by a color separation unit 102. The R, G, 
and B signals are subjected to white balance ad- 
justment by a white balance unit 1 11 on the basis 

10 of color temperature information obtained from a 
white balance sensor (AWB) 120. An output from 
the white balance unit 111 is subjected to 7 correc- 
tion by a 7 correction unit 112. An output from the 
7 correction unit 112 is A/D-converted by an A/D 

75 (analog-to-digital) converter 103. 

A luminance signal is switched by a switch 
circuit (SWY) 126 and is rearranged in a read 
order. The rearranged luminance signal is extracted 
as a high-frequency component Y H of the lumi- 

20 nance signal by a band-pass filter (BPF) 116. The 
high-frequency component Y H of the luminance 
signal is added by an adder 117 to a low-frequency 
component Y L of the luminance signal obtained by 
a technique to be described later. A sum signal is 

25 converted by a D/A (digital-to-analog) converter 
118, and an analog signal is output from the D/A 
converter 118. 

Meanwhile, a G (7th power) signal of outputs 
from the A/D converter 103 is separated by a 

30 switch (SW) 128 into G1 ( 7 th power) and G 2 (7th 
power) signals located at the illustrated positions in 
Fig. 12. This operation can be performed by op- 
erating the switch 128, e.g., every horizontal scan- 
ning period. The separated G1 (7th power) and G2 

35 (7th power) signals are input together with an R 
(7th power) signal and a B (7th power) signal to 
interpolation filters 106, 107, 108, and 109, respec- 
tively, thereby obtaining coincident R (7th power), 
Gi (7th power), G2 (7th power), and B (7th power) 

40 signals. In addition to the coincidence operation by 
interpolation in the interpolation filters 106 to 109, 
linear processing such as two-dimensional low- 
pass filtering and edge emphasis are performed. 
Since these processing operations are linear pro- 

45 cessing operations, they can be performed after 
addition and matrix processing (both will be de- 
scribed in detail later). 

The coincident R (7th power) and Gt (7th 
power) signals are subtracted from each other by 

so an adder 129, and an (R (7th power) - G1 ( T th 
power)) signal is output from the adder 129. Simi- 
larly, the B (7th power) and G2 (7th power) signals 
are subtracted from each other by an adder 130, 
and a (B (7th power) - G 2 (7th power)) signal is 

55 output from the adder 130. These output signals 
are input to a color difference matrix processor 
113, and the following matrix operation is per- 
formed to derive color difference signals R-Y and 
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B-Y. 
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Assume that a monochrome object at a posi- 
tion (1/2P H ,0) in the frequency space is picked up 
by the image pickup element 101. This object 
represents vertical fringes having a period 2P H - For 
this object, conditions R (7th power) = Gi (7th 
power) and B (7th power) = G2 (7th power) are 
established. The color difference signals, i.e., the 
(Ft (7th power) - G1 (7th power)) signal and the (B 
(7th power) - G 2 (7th power)) signal respectively 
output from the adders 129 and 130 become zero. 
Therefore, the color difference signals R-Y and B-Y 
output from the color difference matrix processor 
113 become zero and are not output. This indicates 
that the carrier of the color difference signal at the 
frequency point (1/2P H ,0) is canceled. In other 
words, the carrier of the R (7th power) signal is set 
in phase with that of the G1 (7th power), and the 
carrier of the B (7th power) is also in phase with 
that of the G2 (7th power) signal at the frequency 
point (1/2P H ,0). Since these difference signals, i.e., 
the (R (7th power) - G1 (7th power)) signal and the 
(B (7th power) - G2 (7th power)) signal, can cancel 
their carriers at this frequency, the carriers of the 
color difference signals are not generated. These 
color difference signals are converted into analog 
signals by D/A converters 114 and 115, and the 
analog signals are output. A low-frequency compo- 
nent of the luminance signal is generated by a 
luminance signal generation circuit 127 in accor- 
dance with outputs from the interpolation filters 
106, 107, 108, and 109 as follows: 

Y L = 0.30R (7th power) + [aGi ( 7 th power) + 
0G2 (7th power)] + 0.1 1B (7th power) 
for a + & - 0.59 

The low-frequency component is added to the 
high-frequency component Y H by the adder 117. 
An output from the adder 1 1 7 is D/A-converted by 
the D/A converter 118, and an analog signal is 
output. Since the color difference signals R-Y and 
B-Y and the low-frequency component Y L of the 
luminance signal generally have bandwidths suffi- 
ciently narrower than the luminance signal Y, oper- 
ations of the adders 129 and 130, the color dif- 
ference matrix processor 113, the luminance signal 
generation circuit 127, and the like for the interpo- 
lated coincident R (7th power), G1 (7th power), G 2 
(7th power), and B (7th power) signals may be 
performed using a clock having a lower clock rate 



than that of the luminance signal Y by extraction or 
interlacing. 

As described above, according to this embodi- 
ment, since stripe filters are not used, the moire 
5 can be minimized. In addition, since complemen- 
tary color filters are not used, a high S/N ratio can 
be obtained. Since the color difference signal car- 
riers at the frequency point (1/2P H ,0) can be can- 
celed, a higher resolution can be obtained. 

10 The fourth embodiment of the present inven- 

tion will be described below. 

A filter array as in the first embodiment (Fig. 
4A) is arranged in an image pickup element 101. 
The overall arrangement of the fourth embodiment 

75 is substantially the same as that of Fig. 11, except 
that a G (7th power) signal output from an A/D 
converter 103 is separated into G1 (7th power) and 
G2 (7th power) signals located at positions shown 
in Fig. 13 by a switch 128. 

20 Assume that a monochrome object at a posi- 

tion (0,1 /2P V ) in the frequency space is picked up 
by the image pickup element 101. This object 
represents horizontal fringes having a period 2P V . 
For this object, conditions R (7th power) = Gi (7th 

25 power) and B (7th power) = G2 (7th power) are 
established. The color difference signals, i.e., the 
(R (7th power) - G1 (7th power)) signal and the (B 
(7th power) - G2 (7th power)) signal respectively 
output from adders 129 and 130 become zero. 

30 Therefore, the color difference signals R-Y and B-Y 
output from a color difference matrix processor 113 
become zero and are not output. This indicates that 
the carrier of the color difference signal at the 
frequency point (0,1 /2P V ) is canceled. In other 

35 words, the carrier of the R (7th power) signal is set 
in phase with that of the Gi (7th power), and the 
carrier of the B (7th power) is also in phase with 
that of the G2 (7th power) signal at the frequency 
point (0,1 /2P V ). Since these difference signals, i.e., 

40 the (R (7th power) - Gt (7th power)) signal and the 
(B ( 7 th power) - G2 (7th power)) signal, can cancel 
their carriers at this frequency, the carriers of the 
color difference signals are not generated. 

When an output signal obtained from the ap- 

45 paratus shown in Fig. 11 is to be recorded in an 
analog form, D/A converters 118, 114, and 115 are 
necessary. However, when this output signal is 
recorded in a digital form in a magnetic medium, 
an optomagnetic medium, an E 2 PROM (Electrically 

so Erasable PROM), or the like, the above D/A con- 
verters can be omitted. 

The fifth embodiment of the present invention 
will be described below. 

Fig. 14 is a block diagram showing a signal 

55 processing section of a "color image pickup ap- 
paratus" of the fifth embodiment of the present 
invention. Y, R, and B filters having a Beyer array 
shown in Fig. 4B are arranged in an image pickup 
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element (sensor) 401. An image signal read from 
the image pickup element 401 pixel by pixel is 
separated into Y, R, and B signals by a color 
separation unit 402. The Y, R, and B signals are 
subjected to white balance adjustment by a white 
balance unit 41 1 on the basis of color temperature 
information obtained from a white balance sensor 
(AWB) 420. An output from the white balance unit 
411 is subjected to 7 correction by a 7 correction 
unit 412. An output from the y correction unit 412 
is A/D-converted by an A/D (analog-to-digital) con- 
verter 403. 

A luminance signal is obtained as follows. An 
offset sampling structure of a Y ( 7 th power) signal 
is two-dimensionally interpolated by an interpola- 
tion filter 425, and an output from the interpolation 
filter 425 is D/A-converted by a D/A converter 418. 
An analog signal is thus output from the D/A con- 
verter 418. In addition to the coincidence operation 
by interpolation in the interpolation filter 425, linear 
processing such as two-dimensional low-pass filter- 
ing and edge emphasis are performed. 

Meanwhile, a Y ( 7 th power) signal of outputs 
from the A/D converter 403 is separated by a 
switch 428 into Yi ( 7 th power) and Y 2 ( 7 th power) 
signals located at positions shown in Fig. 15. This 
operation can be performed by operating the 
switch 428, e.g., every horizontal scanning period. 
The separated Yi ( 7 th power) and Y2 ( 7 th power) 
signals are input together with an R ( 7 th power) 
signal and a B ( 7 th power) signal to interpolation 
filters 406, 407, 408, and 409, respectively, thereby 
obtaining coincident R ( 7 th power), Yi ( 7 th power), 
Y 2 ( 7 th power), and B ( 7 th power) signals. In addi- 
tion to the coincidence operation by interpolation in 
the interpolation filters 406 to 409, linear process- 
ing such as two-dimensional low-pass filtering and 
edge emphasis are performed. Since these pro- 
cessing operations are linear processing oper- 
ations, they can be performed after addition (to be 
described in detail later). 

The coincident R ( 7 th power) and Yi ( 7 th pow- 
er) signals are subtracted from each other by an 
adder 429, and an R-Y signal is output from the 
adder 429. Similarly, the B ( 7 th power) and Y 2 ( 7 th 
power) signals are subtracted from each other by 
an adder 430, and a B-Y signal is output from the 
adder 430. 

Assume that a monochrome object at a posi- 
tion (1/2P H ,0) in the frequency space is picked up 
by the image pickup element 401. This object 
represents vertical fringes having a period 2P H - For 
this object, the signals R-Y and B-Y respectively 
output from the adders 429 and 430 become zero 
and are not output. This indicates that the carrier of 
the color difference signal at the frequency point 
(1/2P H ,0) is canceled. 

The sixth embodiment of the present invention 
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will be described below. 

An image pickup element has a Beyer array 
(Fig. 4B) as in the fifth embodiment. The overall 
arrangement of the sixth embodiment is substan- 

5 tially the same as that of the fifth embodiment 
shown in Fig. 14, except that a Y ( 7 th power) signal 
output from an A/D converter 403 is separated into 
Yi ( 7 th power) and Y 2 ( 7 th power) signals located 
at positions shown in Fig. 16 by a switch 428. 

70 Assume that a monochrome object at a posi- 

tion (0,1 /2P V ) in the frequency space is picked up 
by an image pickup element 401. This object re- 
presents horizontal fringes having a period 2P V . For 
this object, the signals R-Y and B-Y respectively 

15 output from adders 429 and 430 become zero and 
are not output. This indicates that the carrier of the 
color difference signal at the frequency point 
(0,1 /2P V ) is canceled. 

The present invention will be described in de- 

20 tail with reference to still another embodiment. 

Fig. 17 is a block diagram showing a "color 
image pickup apparatus" as the seventh embodi- 
ment of the present invention. R, G, and B filters 
(filter array) having a Beyer array shown in Fig. 4A 

25 are arranged in an image pickup element (sensor) 
101. An image signal read from the image pickup 
element 101 pixel by pixel is separated into R, G, 
and B signals by a color separation unit 102. The 
R, G, and B signals are subjected to white balance 

30 adjustment by a white balance unit 111 on the 
basis of color temperature information obtained 
from a white balance sensor (AWB) 120. An output 
from the white balance unit 111 is subjected to 7 
correction by a 7 correction unit 112. An output 

35 from the 7 correction unit 112 is A/D-converted by 
an A/D (analog-to-digital) converter 103. 

A luminance signal is switched by a switch 
circuit (SWY) 126 and is rearranged in a read 
order. The rearranged luminance signal is extracted 

40 as a high-frequency component Y H of the lumi- 
nance signal by a band-pass filter (BPF) 116. The 
high-frequency component Y H of the luminance 
signal is added by an adder 1 17 to a low-frequency 
component Y L of the luminance signal obtained by 

45 a technique to be described later. A sum signal is 
converted by a D/A (digital-to-analog) converter 
118, and an analog signal is output from the D/A 
converter 118. 

Meanwhile, a G ( 7 th power) signal of outputs 

50 from the A/D converter 103 is separated by a 
switch (SW) 128 into Gi ( 7 th power) and G 2 ( 7 th 
power) signals located at positions shown in Fig. 
18. This operation can be performed by operating 
the switch 128, e.g., every horizontal scanning pe- 

55 riod. The separated G1 ( 7 th power) and G 2 ( 7 th 
power) signals are input together with an R ( 7 th 
power) signal and a B ( 7 th power) signal to inter- 
. polation filters 106, 107, 108, and 109, respectively, 

14 




25 EP 0 

thereby obtaining coincident R (7th power), Gi (7th 
power), G2 (7th power), and B (7th power) signals. 
In addition to the coincidence operation by inter- 
polation in the interpolation filters 106 to 109, linear 
processing such as two-dimensional low-pass filter- 
ing and edge emphasis are performed. Since these 
processing operations are linear processing oper- 
ations, they can be performed after addition and 
matrix processing (both will be described in detail 
later). 

The coincident R (7th power) and Gi (7th 
power) signals are subtracted from each other by 
an adder 129, and an (R (7th power) - Gi (7th 
power)) signal is output from the adder 129. Simi- 
larly, the B (7th power) and G2 (7th power) signals 
are subtracted from each other by an adder 130, 
and a (B (7th power) - G2 (7th power)) signal is 
output from the adder 130. These output signals 
are input to a color difference matrix processor 
113, and the following matrix operation is per- 
formed to derive color difference signals R-Y and 
B-Y. 



R- 


Y 




0 .70-0 . 11 


B- 


Y 




-0 . 30+0 . 89 



Assume that a monochrome object at a posi- 
tion (1/2P H ,0) in the frequency space is picked up 
by the image pickup element 101. This object 
represents vertical fringes having a period 2P H . For 
this object, conditions R (7th power) = Gi {7th 
power) and B ( 7 th power) = G2 (7th power) are 
established. The color difference signals, i.e., the 
(R (7th pc-wer) - Gi (7th power)) signal and the (B 
(7th power) - G2 (7th power)) signal respectively 
output from the adders 129 and 130 become zero. 
Therefore, the color difference signals R-Y and B-Y 
output from the color difference matrix processor 
113 become zero and are not output. This indicates 
that the carrier of the color difference signal at the 
frequency point (1/2P H ,0) is canceled. In other 
words, the carrier of the R (7th power) signal is set 
in phase with that of the Gi (7th power), and the 
carrier of the B (7th power) is also in phase with 
that of the G2 (7th power) signal at the frequency 
point (1/2P H ,0). Since these difference signals, i.e., 
the (R (7th power) - Gi (7th power)) signal and the 
(B (7th power) - G2 (7th power)) signal, can cancel 
their carriers at this frequency, the carriers of the 
color difference signals are not generated. Color 
difference signal carriers are not generated at a 
position (-1/2P H ,0) symmetrical with the point 
(1/2H P ,0) about the f v -axis for the same reason. 
These color difference signals are converted into 
analog signals by D/A converters 114 and 1 15, and 
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the analog signals are output. A low-frequency 
component of the luminance signal is generated by 
a luminance signal generation circuit 127 in accor- 
dance with outputs from the interpolation filters 
5 106, 107, 108, and 109 as follows: 

Y L = 0.30R ( 7 th power) + [aGi (7th power) + 
j8G 2 (7th power)] + 0.1 1B ( 7 th power) 
for a + 0 = 0.59 

10 

The low-frequency component is added to the 
high-frequency component Y H by the adder 117. 
An output from the adder 1 1 7 is D/A-converted by 
the D/A converter 118, and an analog signal is 

75 output. Since the color difference signals R-Y and 
B-Y and the low-frequency component Y L of the 
luminance signal generally have bandwidths suffi- 
ciently narrower than the luminance signal Y, oper- 
ations of the adders 129 and 130, the color dif- 

20 ference matrix processor 113, the luminance signal 
generation circuit 127, and the like for the interpo- 
lated coincident R (7th power), Gi (7th power), G2 
(7th power), and B (7th power) signals may be 
performed using a clock having a tower clock rate 

25 than that of the luminance signal Y by extraction or 
interlacing. 

An optical low-pass filter 1 shown in Fig. 17 will 
be described below. 

Fig. 19 shows an arrangement of the optical 

30 low-pass filter 1 of this embodiment. Referring to 
Fig. 19, an optical low-pass filter 300 comprises an 
optical member 301 for splitting an incident beam 
into two beams spaced apart from each other by a 
distance D1 in a direction rotated through 61 coun- 

35 terclockwise with respect to the scanning direction, 
and an optical member 302 for splitting an incident 
beam into two beams spaced apart from each 
other by a distance D 2 in a direction rotated 
through 6 2 clockwise with respect to the scanning 

40 direction. The optical member 301 comprises a 
birefringent plate 303 whose direction of projection 
to a plane parallel to an image plane of the optical 
axis has an angle fli in the counterclockwise direc- 
tion with respect to the scanning direction. The 

45 optical member 302 comprises a \/4 plate 304 for 
converting a linearly polarized component into a 
circularly polarized component and a birefringent 
plate 305 whose direction of projection to a plane 
parallel to an image plane of the optical axis has an 

so angle 62 in the clockwise direction with respect to 
the scanning direction. These components satisfy 
the following conditions: 

O.8P H P v /|PHSin0i +P v cos0i| £ Q^ £ 

55 1 .2 H P v /|PHSin0i + P v cos0i | 
0 £ 6i £ tt/2 (i) 

O.8P H P v /|PHSin0 2 +P v cos0 2 | £ D 2 £ 

15 
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1.2 H P v /|PHSin0 2 +P v cos0 2 | 
0 < 82 £ n/2 (ii) 



When the values Di and D 2 exceed the lower 
limits of inequalities (i) and (ii), folded distortion is 
increased, resulting in inconvenience. When these 
values exceed the upper limits, the resolution is 
decreased, resulting in inconvenience. When an 
application field is limited and only a frequency 
component having a specific direction is an issue, 
either the optical member 301 or 302 can be used. 
At this time, the \/4 plate 304 constituting the 
optical member 302 need not be used, so that the 
optical member 302 comprises only the birefrin- 
gent plate 305. Fig. 20 shows spatial frequency 
characteristics of this optical low-pass filter 300. 
More specifically, Fig. 20 shows a case under the 
following conditions: 

6] = 62 = t/4 

D1 = D 2 = V2P H P V /(P H + Pv) 

At this time, as indicated by dotted lines in Fig. 
20, all the color difference signal carriers at posi- 
tions (±1/2P H ,t1/2P v ) on the spatial frequency 
plane (f H ,M are trapped, and the color difference 
signal carriers at positions (0 t ±1/2P v ) can be suffi- 
ciently suppressed, thereby obtaining a good im- 
age having minimum folded distortion. If 0i = 62 
= tt/4 is established, the optical low-pass filter may 
be arranged as indicated as a filter 500 shown in 
Fig. 21. More specifically, the filter 500 comprises 
an optical member 501 and an optical member 
502. The optical member 501 comprises a birefrin- 
gent plate 503 having a direction of projection to a 
surface parallel to the image plane of the optical 
axis at an angle of W4 in the counterclockwise 
direction with respect to the scanning direction and 
a birefringent plate 504 having a direction of pro- 
jection to a surface parallel to the image plane of 
the optical axis at an angle of tt/4 in the clockwise 
direction with respect to the scanning direction. 
The optical member 502 comprises a birefringent 
plate 505 having a direction of projection to a 
surface parallel to the image plane of the optical 
axis in the scanning direction. The beam distances 
of the birefringent plates 503, 504, and 505 are 
given as Di/V2, D1/V2, and D 2t respectively. 

With the above arrangement, a beam incident 
on the first optical member 501 is split into two 
beams spaced apart from each other by a distance 
Di by the polarization behavior of the birefringent 
plate. The propagation direction of beams is a tt/2 
clockwise direction with respect to the scanning 
direction. The optical low-pass filter shown in Fig. 
19 can have the same spatial frequency char- 
acteristics as those for B\ = 62 = tt/4. 
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The optical low-pass filter 1 may have an ar- 
rangement shown in Fig. 22. This optical low-pass 
filter 1 comprises an optical member 601 consist- 
ing of a birefringent plate for splitting a beam in a 

5 tt/4 counterclockwise direction with respect to the 
scanning direction, an optical member 602 consist- 
ing of a birefringent plate for splitting a beam in a 
7r/2 counterclockwise direction with respect to the 
scanning direction, and an optical member 603 

10 consisting of a birefringent plate for splitting a 
beam in a tt/4 clockwise direction with respect to 
the scanning direction. 

The beam splitting widths of the optical mem- 
bers 601, 602, and 603 are defined as Di , D 2 , and 

75 D 3 : 

Di = D 3 = V2P H Pv/<P H + Pv) 
D 2 = P v 

20 The spatial frequency characteristics of an op- 

tical low-pass filter are given as shown in Fig. 23. 
AH the color difference signal carriers at points 
(±1/2P H ,±1/2P V ) and (0,±1/2P V ) are trapped, and 
folded distortion can be properly suppressed. 

25 As described above, according to this embodi- 

ment, since the color filters constituting the Beyer 
array are used, the moire can be minimized and a 
high S/N ratio can be obtained. In addition, since 
the optical low-pas filter and the signal processing 

30 means which are suitable for the Beyer array are 
used, a higher resolution can be obtained. 

The eighth embodiment of the present inven- 
tion will be described below. 

An image pickup element 101 has color filters 

35 identical to those of the first embodiment shown in 
Fig. 4A, and the overall arrangement of the eighth 
embodiment is the same as that of Fig. 17. A G 
(7th power) signal is separated into Gi (7th power) 
and G 2 (7th power) signals located at positions 

40 shown in Fig. 24 by a switch 128. Assume a 
monochrome object at a position (0,1 /2P V ) in the 
frequency space is picked up by the image pickup 
element 101. This object represents horizontal 
fringes having a period 2P V . For this object, the 

45 color difference signals, i.e., the (R (7th power) - 
G) (7th power)) signal and the (B (7th power) - G 2 
(7th power)) signal respectively output from adders 
129 and 130 become zero. Therefore, the color 
difference signals R-Y and B-Y output from a color 

50 difference matrix processor 113 become zero and 
are not output. This indicates that the carrier of the 
color difference signal at the frequency point 
(0,1 /2P V ) is canceled. In other words, the carrier of 
the R (7th power) signal is set in phase with that of 

55 the G1 (7th power), and the carrier of the B (7th 
power) is also in phase with that of the G 2 (7th 
power) signal at the frequency point (0,1 /2P V ). 
Since these difference signals, i.e., the (R (7th 

16 
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power) - Gi ( 7 th power)) signal and the (B ( 7 th 
power) - G2 (7th power)) signal, can cancel their 
carriers at this frequency, the carriers of the color 
difference signals are not generated. Color dif- 
ference signal carriers are not generated at a posi- 
tion (0,-1/2P v ) symmetrical with the point (0,1/2P V ) 
about the f H -axis for the same reason. When an 
output signal obtained from the apparatus shown in 
Fig. 17 is to be recorded in an analog form, D/A 
converters 118, 114, and 115 are necessary. How- 
ever, when this output signal is recorded in a digital 
form in a magnetic medium, an optomagnetic me- 
dium, an E 2 PROM (Electrically Erasable PROM), or 
the like, the above D/A converters can be omitted. 
The spatial frequency characteristics obtained by 
employing the optical low-pass filter shown in Fig. 
19 or 21 in this image pickup apparatus are shown 
in Fig. 25. All the color difference signal carriers at 
points (±1/2P H ,±1/2P V ) on the spatial frequency 
plane (Wv) are trapped, and color difference signal 
carriers at points (±1/2P H ,0) are sufficiently sup- 
pressed. Folded distortion can be properly sup- 
pressed. 

The optical low-pass filter may have an ar- 
rangement shown in Fig. 26. This optical low-pass 
filter 150 comprises an optical member 151 con- 
sisting of a birefringent plate for splitting a beam in 
a tt/4 counterclockwise direction with respect to the 
scanning direction, an optical member 152 consist- 
ing of a birefringent plate for splitting a beam in a 
direction parallel to the scanning direction, and an 
optical member 153 consisting of a birefringent 
plate for splitting a beam in a ?r/4 clockwise direc- 
tion with respect to the scanning direction. The 
beam splitting widths of the optical members 151, 
152, and 153 are defined as Di , D 2 , and D 3 : 

D1 = D3 = V2P H P V /(P H + P v ) 
D 2 = P H 

The spatial frequency characteristics of an op- 
tical low-pass filter 150 are given as shown in Fig. 
27. All the color difference signal carriers at points 
(±1/2P H ,±1/2P V ) and (±1/2P H ,0) are trapped, and 
folded distortion can be properly suppressed. 

Fig. 28 is a block diagram of a "color image 
pickup apparatus" according to a ninth embodi- 
ment. A beam from an object to be photographed 
is incident on an image pickup element (sensor) 
101 through an optical low-pass filter 1 by a focus- 
ing optical system (not shown). 

The optical low-pass filter 1 has an arrange- 
ment shown in Fig. 19, 21, or 22 and properly 
suppresses folded distortion. The image pickup 
element (sensor) 101 has YRB filters constituting a 
Beyer array shown in Fig. 4B. An image signal 
read from the image pickup element 101 pixel by 
pixel is separated into Y, R, and B signals by a 



color separation unit 102. The Y, R, and B signals 
are subjected to white balance adjustment by a 
white balance unit 1 1 1 on the basis of color tem- 
perature information obtained from a white balance 
5 sensor 120. An output from the white balance unit 
111 is subjected to 7 correction by a 7 correction 
unit 112. An output from the 7 correction unit 112 
is A/D-converted by an A/D (analog-to-digital) con- 
verter 103. A luminance signal is obtained as fol- 

10 lows. An offset sampling structure of a Y (7th 
power) signal is two-dimensionally interpolated by 
an interpolation filter 205, and an output from the 
interpolation filter 205 is D/A-converted by a D/A 
converter 118. An analog signal is thus output from 

75 the D/A converter 118. In addition to the coinci- 
dence operation by interpolation in the interpolation 
filter 205, processing such as two-dimensional low- 
pass filtering and edge emphasis are performed. 
Meanwhile, a Y (7th power) signal of outputs from 

20 the A/D converter 103 is separated by a switch 128 
into Y1 ( 7 th power) and Y 2 (7th power) signals 
located at the illustrated positions shown in Fig. 29. 
This operation can be performed by operating the 
switch 128, e.g., every horizontal scanning period. 

25 The separated Y1 (7th power) and Y 2 (7th power) 
signals are input together with an R (7th power) 
signal and a B (7th power) signal to interpolation 
filters 206, 207, 208, and 209, respectively, thereby 
obtaining coincident R (7th power), Yi (7th power), 

30 Y 2 (7th power), and B (7th power) signals. In addi- 
tion to the coincidence operation by interpolation in 
the interpolation filters 206 to 209, linear process- 
ing such as two-dimensional low-pass filtering and 
edge emphasis are performed. Since these pro- 

35 cessing operations are linear processing oper- 
ations, they can be performed after addition (to be 
described in detail later). The coincident R (7th 
power) and Yi (7th power) signals are subtracted 
from each other by an adder 129, and an R-Y 

40 signal is output from the adder 129. Similarly, the B 
( 7 th power) and Y 2 (7th power) signals are sub- 
tracted from each other by an adder 130, and a B- 
Y signal is output from the adder 130. 

Assume a monochrome object at a position 

45 (1/2P Hf 0) in the frequency space is picked up by 
the image pickup element 101. This object repre- 
sents vertical fringes having a period 2P H . For this 
object, the signals R-Y and B-Y respectively output 
from the adders 129 and 130 become zero and are 

50 not output. This indicates that the carrier of the 
color difference signal at the frequency point 
(1/2P H ,0) is canceled. In other words, the carrier of 
the R (7th power) signal is set in phase with that of 
the Yt (7th power), and the carrier of the B ( 7 th 

55 power) is also in phase with that of the Y 2 (7th 
power) signal at the frequency point (1/2P H ,0). 
Since these R-Y and B-Y can cancel their carriers 
at this frequency, the carriers of the color dif- 

17 
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ference signals are not generated. These color 
difference signals are D/A-converted into analog 
signals by D/A converters 114 and 115. Color dif- 
ference signal carriers are not generated at a posi- 
tion (-1/2P H ,0) symmetrical with the point (1/2H P ,0) 
about the f v -axis for the same reason. Since the 
color difference signals R-Y and B-Y generally 
have bandwidths sufficiently narrower than the lu- 
minance signal Y, operations of the adders 129 and 
130, and the like for the interpolated and coincided 
R (-yth power), Yi ( 7 th power), Y 2 (7th power), and 
B (yth power) signals may be performed using a 
clock having a lower clock rate than that of the 
luminance signal Y by extraction or interlacing. In 
this embodiment, unlike in the seventh and eighth 
embodiments, the carriers of not the color dif- 
ference signals but the luminance signals appear at 
positions (±1/2P H ,±1/2P V ) on the spatial frequency 
plane (f H ,fv)The optical low-pass filters 300, 500, 
and 600 shown in Figs. 19, 21, and 22 have fre- 
quency characteristics represented by the dotted 
lines in Figs. 20 and 23, so that the above carriers 
can be trapped and folded distortion is also prop- 
erly suppressed. 

The tenth embodiment of the present invention 
will be described below. 

An image pickup element 101 has color filters 
identical to those of the ninth embodiment shown 
in Fig. 4B, and the overall arrangement of the tenth 
embodiment is the same as that of Fig. 28. A Y 
(7th power) signal is separated into Y1 (7th power) 
and Ys(7th power) signals located at positions 
shown in Fig. 30 by a switch 128. 

Assume a monochrome object at a position 
(0,1 /2P V ) in the frequency space is picked up by 
the image pickup element 101. This object repre- 
sents horizontal fringes having a period 2P V . For 
this object, the color difference signals R-Y and B- 
Y respectively output from adders 129 and 130 
become zero. Therefore, the color difference sig- 
nals R-Y and B-Y output from the adders 129 and 
130 become zero and are not output. This indicates 
that the carrier of the color difference signal at the 
frequency point (0,1/2P V ) is canceled. In other 
words, the carrier of the R (7th power) signal is set 
in phase with that of the Yi (7th power), and the 
carrier of the B (7th power) is also in phase with 
that of the Y 2 (7th power) signal at the frequency 
point (0,1 /2P V ) Since these difference signals R-Y 
and B-Y can cancel their carriers at this frequency, 
the carriers of the color difference signals are not 
generated. When an output signal obtained from 
the apparatus shown in Fig. 28 is to be recorded in 
an analog form, D/A converters 118, 114, and 115 
are necessary. However, when this output signal is 
recorded in a digital form in a magnetic medium, 
an optomagnetic medium, an E 2 PROM (Electrically 
Erasable PROM), or the like, the above D/A con- 



verters can be omitted. 

The optical low-pass filter 1 may comprise an 
arrangement shown in Fig. 19, 21, or 26. The 
spatial frequency characteristics of such a filter are 

5 excellent to properly suppress folded distortion, as 
shown in Figs. 25 and 27. 

In addition, the color difference signals may be 
formed as follows. That is, as shown in Fig. 31, 
separation of the G (7th power) signal by the 

10 switch 128 can be switched between timings 
shown in Fig. 18 and Fig. 24 in accordance with a 
luminance signal of an object. This arrangement 
will be described as the eleventh embodiment. 
Selection of the timing of Fig. 18 or 24 is deter- 

75 mined by a decision circuit 131 (to be described 
later). Similarly, as shown in Fig. 32, separation of 
the Y (7th power) signal by a switch 203 is per- 
formed by switching between timings shown in 
Figs. 29 and 30 in accordance with a luminance 

20 signal of the object, as shown in Fig. 32. This 
arrangement will be described as the twelfth em- 
bodiment. 

Selection of either timing is determined by a 
decision circuit 204 (to be described later). Ar- 

25 rangements and operations of the decision circuit 
131 (Fig. 31) and 204 (Fig. 32) will be described 
below. The luminance signal output from the switch 
circuit 126 or the interpolation filter 205 is filtered 
by a band-pass filter (H-BPF) 31 in the horizontal 

30 direction to extract a horizontal high-frequency 
component. This extracted output is input to a 
comparator (comp) 32 and is compared with a 
predetermined threshold value. When it is decided 
that the input signal level is higher than the thresh- 

35 old level and the horizontal high-frequency compo- 
nent value is large, the timing in Fig. 18 is selected 
to cancel the carrier of the horizontal color dif- 
ference signal. Otherwise, the timing (Fig. 24) for 
canceling the vertical color difference signal carrier 

40 is selected. 

The decision circuits 131 and 204 may be 
arranged as in the circuit of Fig. 34. More specifi- 
cally, the luminance signal output from the switch 
circuit 126 or the interpolation filter 205 is filtered 

45 by a band-pass filter (V-BPF) 61 in the vertical 
direction to extract a vertical high-frequency com- 
ponent. This extracted output is input to a com- 
parator (comp) 62 and is compared with a pre- 
determined threshold value. When it is decided that 

50 the input signal level is higher than the threshold 
level and the vertical high-frequency component 
value is large, the timing shown in Fig. 24 is 
selected to cancel the carrier of the vertical color 
difference signal. Otherwise, the timing of Fig. 18 

55 for canceling the horizontal color difference signal 
carrier is selected. In this manner, since the car- 
riers to be canceled can be changed in accordance 
with the horizontal or vertical frequency compo- 

18 
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nents of the object in the eleventh or twelfth em- 
bodiment, a good image having minimum folded 
distortion can be obtained. If the optical low-pass 
filter 1 has arrangements shown in Figs. 19, 21, 22, 
and 26, folded distortion can be properly sup- s 
pressed. 

As described above, since the color filters con- 
stituting the Beyer array are used in the image 
pickup element, appropriate signal processing is 
performed. There is therefore provided a color im- w 
age pickup apparatus capable of producing an im- 
age having a high resolution, reduced moire, and a 
high S/N ratio. 

The present invention will be described in de- 
tail with reference to still another embodiment 15 
hereinafter. 

Fig. 35 is a block diagram showing a "color 
image pickup apparatus" as the thirteenth embodi- 
ment of the present invention. R, G, and B filters 
(filter array) having a Beyer array shown in Fig. 4A 20 
are arranged in an image pickup element (sensor) 
101. An image signal read from the image pickup 
element 101 pixel by pixel is separated into R, G, 
and B signals by a color separation unit 102. The 
R, G, and B signals are subjected to white balance 25 
adjustment by a white balance unit 111 on the 
basis of color temperature information obtained 
from a white balance sensor (AWB) 120. An output 
from the white balance unit 111 is subjected to 7 
correction by a 7 correction unit 112. An output 30 
from the 7 correction unit 112 is A/D-converted by 
an A/D (analog-to-digital) converter 103. 

A luminance signal is switched by a switch 
circuit (SWY) 126 and is rearranged in a read 
order. The rearranged luminance signal is read as 35 
a luminance signal Y s including a high-frequency 
component. This luminance signal Y s is added by 
an adder 117 to signals representing the products 
between constants and a first difference signal, i.e., 
an (R (7th power) - G1 ( 7 th power)) signal and a 40 
second difference signal, i.e., a (B (7th power) - G 2 
(7th power)) signal. An output from the adder 117 
is D/A-converted by a D/A (digital-to-analog) con- 
verter 118. 

Meanwhile, a G (7th power) signal of outputs 45 
from the A/D converter 103 is separated by a 
switch (SW) 128 into Gi ( 7 th power) and G 2 (7th 
power) signals located at positions shown in Fig. 
36. This operation can be performed by operating 
the switch 128, e.g., every horizontal scanning pe- so 
riod. The separated G1 (7th power) and G 2 (7th 
power) signals are input together with an R ( 7 th 
power) signal and a B ( 7 th power) signal to inter- 
polation filters 106, 107, 108, and 109, respectively, 
thereby obtaining coincident R ( 7 th power), G1 (7th 55 
power), G 2 (7th power), and B ( 7 th power) signals. 
In addition to the coincidence operation by inter- 
polation in the interpolation filters 106 to 109, linear 



processing such as two-dimensional low-pass filter- 
ing and edge emphasis are performed. Since these 
processing operations are linear processing oper- 
ations, they can be performed after addition and 
matrix processing (both will be described in detail 
later). 

The coincident R (7th power) and Gi (7th 
power) signals are subtracted from each other by 
an adder 129, and an (R ( 7 th power) - Gi (7th 
power)) signal is output from the adder 129. Simi- 
larly, the B ( 7 th power) and G 2 (7th power) signals 
are subtracted from each other by an adder 130, 
and a (B ( 7 th power) - G 2 (7th power)) signal is 
output from the adder 130. These output signals 
are input to a color difference matrix processor 
113, and the following matrix operation is per- 
formed to derive color difference signals R-Y and 
B-Y. 



R- 


Y 




0.70-0.11 


B- 


Y 




-0 . 30+0 . 89 



B Y -G 2 T 



Assume a monochrome object at a position 
(1/2P H ,0) in the frequency space is picked up by 
the image pickup element 10.1. This object repre- 
sents vertical fringes having a period 2P H . For this 
object, conditions R (7th power) = Gi ( 7 th power) 
and B (7th power) = G 2 (7th power) are estab- 
lished. The color difference signals, i.e., the (R (7th 
power) - Gi (7th power)) signal and the (B (7th 
power) - G 2 (7th power)) signal respectively output 
from the adders 129 and 130 become zero. There- 
fore, the color difference signals R-Y and B-Y out- 
put from the color difference matrix processor 1 1 3 
become zero and are not output. This indicates that 
the carrier of the color difference signal at the 
frequency point (1/2P H ,0) is canceled. In other 
words, the carrier of the R (7th power) signal is set 
in phase with that of the Gi (7th power), and the 
carrier of the B (7th power) is also in phase with 
that of the G 2 (7th power) signal at the frequency 
point (1/2P H ,0). Since these difference signals, i.e., 
the (R (7th power) - G^ (7th power)) signal and the 
(B ( 7 th power) - G 2 (7th power)) signal, can cancel 
their carriers at this frequency, the carriers of the 
color difference signals are not generated. Color 
difference signal carriers are not generated at a 
position (-1/2P H ,0) symmetrical with the point 
(1/2H P ,0) about the f v -axis for the same reason. 
These color difference signals are D/A-converted 
by D/A converters 114 and 115. 

The first difference signal (R (7th power) - Gi - 
(7th power)) signal and the second difference sig- 
nal (B (7th power) - G 2 (7th power)) are multiplied 
with constants by constant multipliers 132 and 133 



19 



3NGOOCIO: <£P_P602639A2JL* 




35 EP 0 



and are added by an adder 134. An output from 
the adder 134 is added to the luminance signal Y s 
by the adder 117, thereby obtaining a luminance 
signal Y whose spectral characteristics are cor- 
rected. 

The principle of this operation will be described 
below. 

Assume that outputs from the respective pixels 
of the image pickup element 101 are simply syn- 
thesized, and the resultant luminance signal is de- 
fined as Y s . The luminance signal Y s is obtained by 
switching the output signals from the respective 
pixels by means of the switch circuit 126. One of 
the color-separated components, e.g., a G signal 
may be used. To the contrary, the luminance signal 
whose spectral characteristics are corrected to be 
equal to visibility sensitivity is defined as Y L . The 
luminance signal Y L can be formed by linear cou- 
pling of coincident color signals and is defined as 
follows: 

Y L = 5R Y + aGi 7 + 0G 2 7 + *B y 
for 5 + a + & + e = 1 (1) 

In the NTSC scheme, 

5 = 0.30, a + 0 = 0.59, € = 0.1 1 (2) 

The luminance signal Y L has a lower band than 
that of the luminance signal Y s . Of all the lu- 
minance signals Y s , a luminance signal having the 
same band as that of the luminance signal Y L js 
defined as Y SL . 

At this time, the luminance signal Y is cor- 
rected by substituting the low-frequency compo- 
nent Y SL of the luminance signal Y s with the lu- 
minance signal Y L having the correct spectral char- 
acteristics. That is, 

Y - (Y s - Y SL ) + Y|_ = Y|_ + (Y L - Y SL ) (3) 

The luminance signal Y SL can be expressed by 
linear coupling of respective color signals and is 
defined as follows: 

Y SL = sR 7 + tGi Y + uG 2 y + wB* 
for s + t + u + w = 1 (4) 

At this time, terms within the parentheses on 
the right side in equation (3) are defined as follows: 

Y L - Y SL = (5 - s)R 7 - (t - a^ y + {e - w)B T - (u - 
£)G 2 

wherein if S - s = t - a, then s + t = a + 5 
and 5 + a + £ + € = s + t + u+ w(= 1), so 
that 
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0 + € = u + w 
therefore 

5 

e - w = u - /9 

If the following conditions are given: 
w Ci = 5 - s = t - a (5) 
C 2 = e - w = u - fi (6) 
the following equation is derived: 

15 

Y[_ - Y SL = Ci (R y - Qi y ) + C 2 (B Y - G 2 7 ) 

A substitution of this equation into equation (3) 
yields the following: 

20 

Y = Y s + Ci (R Y - Gi Y ) + C 2 (R Y - G 2 Y ) (7) 

The spectral characteristics of the luminance 
signal Y are obtained by adding products of the 

25 constants and the first and second difference sig- 
nals (R (7th power) - G1 (7th power)) and (B (7th 
power) - G 2 (7th power)) to the simply synthesized 
luminance signal Y s . 

When the luminance signal Y s is obtained by a 

30 switching operation of the switch circuit 126, values 
s, t, u, and w in equation (4) satisfy the following 
equation: 

s = t = u = w = 0.25 (8) 

35 

the following relations are derived from equations 
(2), (5), and (6): 

a = 0.20, p = 0.39, 
40 Ci = 0.05, C 2 = -0.14 (9) 

When the luminance signal Y s is obtained us- 
ing the G signal, the values s, t, u, and w are given 
as follows: _ - - . — 

45 

s = w = 0, t = u = 0.5 (10) 

The following relations are then derived from 
equations (2), (5), and (6): 

50 

a = 0.20, 0 = 0.39, 

Ci = 0.30, C 2 = 0.11 (11) 

The luminance signal whose spectral charac- 
55 teristics are corrected as described above is D/A- 
converted by the D/A converter 1 1 8, and an analog 
signal is output from the D/A converter 118. Since 
the color difference signals R-Y and B-Y and the 

20 
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low-frequency component Y L of the luminance sig- 
nal generally have bandwidths sufficiently narrower 
than the luminance signal Y, operations of the 
adders 129 and 130\ the color difference matrix 
processor 113, the adder 117, and the like for the 
interpolated coincident R (7th power), Gi (7th pow- 
er), G2 (7th power), and B ( 7 th power) signals may 
be performed using a clock having a lower clock 
rate than that of the luminance signal Y by extrac- 
tion or interlacing. 

An optical low-pass filter shown in Fig. 35 will 
be described below. 

Fig. 37 shows an arrangement of the optical 
low-pass filter 1 of this embodiment. Referring to 
Fig. 37, an optical low-pass filter 300 comprises an 
optical member 301 for splitting an incident beam 
into two beams spaced apart from each other by a 
distance Di in a direction rotated through 01 coun- 
terclockwise with respect to the scanning direction, 
and an optical member 302 for splitting an incident 
beam into two beams spaced apart from each 
other by a distance D2 in a direction rotated 
through 0 2 clockwise with respect to the scanning 
direction. The optical member 301 comprises a 
birefringent plate 303 whose direction of projection 
to a plane parallel to an image plane of the optical 
axis has an angle 0i in the counterclockwise direc- 
tion with respect to the scanning direction. The 
optical member 302 comprises a X/4 plate 304 for 
converting a linearly polarized component into a 
circularly polarized component and a birefringent 
plate 305 whose direction of projection to a plane 
parallel to an image plane of the optical axis has an 
angle 0 2 in the clockwise direction with respect to 
the scanning direction. These components satisfy 
the following conditions: 

O.8P H P v /|PHSin0i + P v cos0i| £ Di £ 
I^Pv/IPHSinfl! +P v cos0i| 
0 < £h ^ tt/2 (i) 

0.8P H Pv/|PHsine 2 + Pvcos0 2 | < D 2 £ 
1 .2 H P v /iPHSin0 2 + P v cos0 2 | 
0 £ 0 2 ^ W2 (ii) 

When the values Di and D 2 exceed the lower 
limits of inequalities (i) and (ii), folded distortion is 
increased, resulting in inconvenience. When these 
values exceed the upper limits, the resolution is 
decreased, resulting in inconvenience. When an 
application field is limited and only a frequency 
component having a specific direction is an issue, 
either the optical member 301 or 302 can be used. 
At this time, the X/4 plate 304 constituting the 
optical member 302 need not be used, so that the 
optical member 302 comprises only the birefrin- 
gent plate 305. Fig. 38 shows spatial frequency 
characteristics of this optical low-pass filter 300. 



More specifically, Fig. 38 shows a case under the 
following conditions: 

6\ = $ 2 = W4 
5 Di = D 2 = V2P h Pv/(Ph + Pv) 



At this time, as indicated by dotted lines in Fig. 
38, all the color difference signal carriers at posi- 

10 tions (±1/2P H ,±1/2P V ) on the spatial frequency 
plane (f H ,M are trapped, and the color difference 
signal carriers at positions (0,±1/2P V ) can be suffi- 
ciently suppressed, thereby obtaining a good im- 
age having minimum folded distortion. If di = 0 2 

75 = W4 is established, the optical low-pass filter may 
be arranged as indicated as a filter 500 shown in 
Fig. 39. More specifically, the filter 500 comprises 
an optical member 501 and an optical member 
502. The optical member 501 comprises a birefrin- 

20 gent plate 503 having a direction of projection to a 
surface parallel to the image plane of the optical 
axis at an angle of tt/4 in the counterclockwise 
direction with respect to the scanning direction and 
a birefringent plate 504 having a direction of pro- 

25 jection to a surface parallel to the image plane of 
the optical axis at an angle of tt/4 in the clockwise 
direction with respect to the scanning direction. 
The optical member 502 comprises a birefringent 
plate 505 having a direction of projection to a 

30 surface parallel to the image plane of the optical 
axis in the scanning direction. The beam distances 
of the birefringent plates 503, 504, and 505 are 
given as DiA/2, D1A/2, and D 2 , respectively. 

With the above arrangement, a beam incident 

35 on the first optical member 501 is split into two 
beams spaced apart from each other by a distance 
Di by the polarization behavior of the birefringent 
plate. The propagation direction of beams is a tt/2 
clockwise direction with respect to the scanning 

40 direction. The optical low-pass filter shown in Fig. 
37 can have the same spatial frequency char- 
acteristics as those for 0.1 = 0 2 = tt/4. 

The optical low-pass filter 1 may have an ar- 
rangement shown in Fig. 40. This optical low-pass 

45 filter 600 comprises an optical member 601 con- 
sisting of a birefringent plate for splitting a beam in 
a tt/4 counterclockwise direction with respect to the 
scanning direction, an optical member 602 consist- 
ing of a birefringent plate for splitting a beam in a 

50 tt/2 counterclockwise direction with respect to the 
scanning direction, and an optical member 603 
consisting of a birefringent plate for splitting a 
beam in a n/4 clockwise direction with respect to 
the scanning direction. 

55 The beam splitting widths of the optical mem- 

bers 601, 602, and 603 are defined as Di , D 2 , and 
D 3 : 
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Di = D 3 = V2P h P v /(Ph + Pv) 
D 2 = P v 

The spatial frequency characteristics of an op- 
tical low-pass filter are given as shown in Fig. 41. 
All the color difference signal carriers at points 
(±1/2P H ,±1/2P V ) and (0,±1/2P V ) are trapped, and 
folded distortion can be properly suppressed. 

As described above, according to this embodi- 
ment, since the color filters constituting the Beyer 
array are used, the moire can be minimized and a 
high S/N ratio can be obtained. In addition, since 
the optical low-pas filter and the signal processing 
means which are suitable for the Beyer array are 
used, a higher resolution can be obtained. 

The luminance information whose spectral 
characteristics are corrected by a simple technique 
is obtained. 

The fourteenth embodiment of the present in- 
vention will be described below. 

An image pickup element 101 has color filters 
identical to those of the thirteenth embodiment 
shown in Fig. 4A, and the overall arrangement of 
the eighth embodiment is the same as that of Fig. 
42. A G (7th power) signal is separated into G1 (7th 
power) and G2 (7th power) signals located at posi- 
tions shown in Fig. 42 by a switch 128. Assume a 
monochrome object at a position (0,1 /2P V ) in the 
frequency space is picked up by the image pickup 
element 101. This object represents horizontal 
fringes having a period 2P V . For this object, the 
color difference signals, i.e., the (R (7th power) - 
Gi (7th power)) signal and the (B (7th power) - G2 
( 7 th power)) signal respectively output from adders 
129 and 130 become zero. Therefore, the color 
difference signals R-Y and B-Y output from a color 
difference matrix processor 113 become zero and 
are not output. This indicates that the carrier of the 
color difference signal at the frequency point 
(0,1 /2P V ) is canceled. In other words, the carrier of 
the R (7th power) signal is set in phase with that of 
the Gi (7th power), and the carrier of the B (7th 
power) is also in phase with that of the G2 (7th 
power) signal at the frequency point (0,1 /2P V ) Since 
these difference signals, i.e., the (R (7th power) - 
Gi (7th power)) signal and the (B (7th power) - G2 
(7th power)) signal, can cancel their carriers at this 
frequency, the carriers of the color difference sig- 
nals are not generated. Color difference signal car- 
riers are not generated at a position (0,-1 /2P V ) 
symmetrical with the point (0,1 /2P V ) about the f H - 
axis for the same reason. When an output signal 
obtained from the apparatus shown in Fig. 35 is to 
be recorded in an analog form, D/A converters 118, 
114, and 115 are necessary. However, when this 
output signal is recorded in a digital form in a 
magnetic medium, an optomagnetic medium, an 
E 2 PROM (Electrically Erasable PROM), or the like, 
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the above D/A converters can be omitted. The 
spatial frequency characteristics obtained by em- 
ploying the optical low-pass filter shown in Fig. 37 
or 39 in this image pickup apparatus are shown in 

5 Fig. 43. AN the color difference signal carriers at 
points (±1/2P H ,±1/2P V ) on the spatial frequency 
plane (Wv) are trapped, and color difference signal 
carriers at points (±1/2P H ,0) are sufficiently sup- 
pressed. Folded distortion can be properly sup- 

70 pressed. 

The optical low-pass filter may have an ar- 
rangement shown in Fig. 44. This optical low-pass 
filter 150 comprises an optical member 151 con- 
sisting of a birefringent plate for splitting a beam in 

75 a tt/4 counterclockwise direction with respect to the 
scanning direction, an optical member 152 consist- 
ing of a birefringent plate for splitting a beam in a 
direction parallel to the scanning direction, and an 
optical member 153 consisting of a birefringent 

20 plate for splitting a beam in a tt/4 clockwise direc- 
tion with respect to the scanning direction. The 
beam splitting widths of the optical members 151, 
152, and 153 are defined as D1 , D2, and D3: 

25 Di = D 3 = V2P h Pv/(Ph + Pv) 
D 2 = P H 

The spatial frequency characteristics of an op- 
tical low-pass filter 150 are given as shown in Fig. 

30 45. All the color difference signal carriers at points 
<±1/2P H ,±1/2P V ) and (0,±1/2P V ) are trapped, and 
folded distortion can be properly suppressed. 

The luminance information is obtained in the 
same manner as in the thirteenth embodiment. 

35 The fifteenth embodiment of the present inven- 

tion will be described below. 

Fig. 46 is a block diagram of a "color image 
pickup apparatus" according to this embodiment. A 
beam from an object to be photographed is in- 

40 cident on an image pickup element (sensor) 101 
through an optical low-pass filter 1 by a focusing 
optical system (not shown). 

The optical low-pass filter 1 has an arrange- 
ment shown in Fig. 37, 39, or 40 and properly 

45 suppresses folded distortion. The image pickup 
element (sensor) 101 has R, G, and B filters con- 
stituting a Beyer array shown in Fig. 4A. An image 
signal read from the image pickup element 101 
pixel by pixel is separated into R, G, and B signals 

50 by a color separation unit 102. The G, and B 
signals are subjected to white balance adjustment 
by a white balance unit 111 on the basis of color 
temperature information obtained from a white bal- 
ance sensor 120. An output from the white balance 

55 unit 1 1 1 is subjected to 7 correction by a 7 correc- 
tion unit 112. An output from the 7 correction unit 
112 is A/D-converted by an A/D (analog-to-digital) 
converter 103. A luminance signal is obtained as 

22 
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follows. An offset sampling structure of a G (7th 
power) signal is two-dimensionally interpolated by 
an interpolation filter 205, and an output from the 
interpolation filter 205 is D/A-converted by a D/A 
converter 118. An analog signal is thus output from 
the D/A converter 118. In addition to the coinci- 
dence operation by interpolation in the interpolation 
filter 205, processing such as two-dimensional low- 
pass filtering and edge emphasis are performed. 
Meanwhile, a G (7th power) signal of outputs from 
the A/D converter 1 03 is separated by a switch 1 28 
into Gi (7th power) and G2 (7th power) signals 
located at the illustrated positions shown in Fig. 36. 
This operation can be performed by operating the 
switch 128, e.g., every horizontal scanning period. 
The separated G1 (7th power) and G2 (7th power) 
signals are input together with an R (7th power) 
signal and a B ( 7 th power) signal to interpolation 
filters 206, 207, 208, and 209, respectively, thereby 
obtaining coincident R (7th power), G1 (7th power), 
G2 (7th power), and B (7th power) signals. In 
addition to the coincidence operation by interpola- 
tion in the interpolation filters 206 to 209, linear 
processing such as two-dimensional low-pass filter- 
ing and edge emphasis are performed. Since these 
processing operations are linear processing oper- 
ations, they can be performed after addition (to be 
described in detail later). The coincident R (7th 
power) and Gi (7th power) signals are subtracted 
from each other by an adder 129, and an (R (7th 
power) - Gi (7th power)) signal is output from the 
adder 129. Similarly, the B (7th power) and G 2 (7th 
power) signals are subtracted from each other by 
an adder 130, and a (B (7th power) - G 2 (7th 
power)) signal is output from the adder 130. Color 
difference signals R-Y and B-Y are generated by a 
color difference matrix processor 113 as in the 
thirteenth and fourteenth embodiments. 

Assume a monochrome object at a position 
(1/2P H ,0) in the frequency space is picked up by 
the image pickup element 101. This object repre- 
sents vertical fringes having a period 2P H . For this 
object, the (R (7th power) - Gi (7th power)) signal 
and the (B ( 7 th power) - G2 (7th power)) signal 
respectively output from the adders 129 and 130 
become zero and are not output. This indicates that 
the carrier of the color difference signal at the 
frequency point (1/2P H ,0) is canceled. In other 
words, the carrier of the R (7th power) signal is set 
in phase with that of the Gi (7th power), and the 
carrier of the B (7th power) is also in phase with 
that of the G2 (7th power) signal at the frequency 
point (1/2P H ,0). Since these R-Y and B-Y can can- 
cel their carriers at this frequency, the carriers of 
the color difference signals are not generated. 
These color difference signals are D/A-converted 
into analog signals by D/A converters 114 and 115. 
Color difference signal carriers are not generated at 



a position (-1/2P H ,0) symmetrical with the point 
(1/2H P ,0) about the f v -axis for the same reason. 
Since the color difference signals R-Y and B-Y 
generally have bandwidths sufficiently narrower 
5 than the luminance signal Y, operations of the 
adders 129 and 130, and the like for the interpo- 
lated coincident R (7th power), Gi (7th power), G 2 
(7th power), and B (7th power) signals may be 
performed using a clock having a lower clock rate 

10 than that of the luminance signal Y by extraction or 
interlacing. In this embodiment, unlike in the thir- 
teenth and fourteenth embodiments, the carriers of 
not the color difference signals but the luminance 
signals appear at positions (±1/2P H ,±1/2P V ) on the 

75 spatial frequency plane (f H .fv). The optical low-pass 
filters 300, 500, and 600 shown in Figs. 37, 39, and 
40 have frequency characteristics represented by 
the dotted lines in Figs. 38 and 41, so that the 
above carriers can be trapped and folded distortion 

20 is also properly suppressed. 

The luminance information is formed by di- 
rectly using the G signal as Y s in the above de- 
scription of the principle. 

The fifteenth embodiment of the present inven- 

25 tion will be described below. 

An image pickup element 101 has color filters 
identical to those of the fourteenth embodiment 
shown in Fig. 4A, and the overall arrangement of 
the tenth embodiment is the same as that of Fig. 

30 46. A G (7th power) signal is separated into Gi (7th 
power) and G 2 (7th power) signals located at posi- 
tions shown in Fig. 42 by a switch 128. 

Assume a monochrome object at a position 
(0,1 /2P V ) in the frequency space is picked up by 

35 the image pickup element 101. This object repre- 
sents horizontal fringes having a period 2P V . For 
this object, an (R (7th power) - Gi (7th power)) 
signal and a (B (7th power) - G2 (7th power)) signal 
respectively output from adders 129 and 130 be- 

40 come zero. Therefore, the (R (7th power) - Gi (7th 
power)) signal and the (B (7th power) - G 2 (7th 
power)) signal output from the adders 129 and 130 
become zero and are not output. This indicates that 
the carrier of the color difference signal at the 

45 frequency point (0,1/2P V ) is canceled. In other 
words, the carrier of the R (7th power) signal is set 
in phase with that of the Gi (7th power), and the 
carrier of the B (7th power) is also in phase with 
that of the G2 (7th power) signal at the frequency 

50 point (0,1/2P V ). Since these difference signals, i.e., 
the (R ( 7 th power) - Gi (7th power)) signal and the 
(B (7th power) - G2 (7th power)) signal can cancel 
their carriers at this frequency, the carriers of the 
color difference signals are not generated. When 

55 an output signal obtained from the apparatus 
shown in Fig. 46 is to be recorded in an analog 
form, D/A converters 118, 114, and 115 are neces- 
sary. However, when this output signal is recorded 

23 
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in a digital form in a magnetic medium, an op- 
tomagnetic medium, an E 2 PROM (Electrically Eras- 
able PROM), or the like, the above D/A converters 
can be omitted. 

The optical low-pass filter 1 may comprise an 
arrangement shown in Fig. 37, 39, or 44. The 
spatial frequency characteristics are excellent to 
properly suppress folded distortion, as shown in 
Figs. 43 and 45. 

The luminance signal is formed in the same 
manner as in the fourteenth embodiment. 

In addition, the color difference signals may be 
formed as follows. That is, as shown in Fig. 49, 
separation of the G (7th power) signal by the 
switch 128 can be switched between timings 
shown in Figs. 36 and 42 in accordance with a 
luminance signal of an object. This arrangement 
will be described as the sixteenth embodiment. 
Selection of the timing of Fig. 36 or 42 is deter- 
mined by a decision circuit 131 (to be described 
later). Similarly, as shown in Fig. 50, separation of 
the G (7th power) signal by a switch 203 is per- 
formed by switching between timings shown in 
Figs. 36 and 42 in accordance with a luminance 
signal of the object. This arrangement will be de- 
scribed as the seventeenth embodiment. 

Selection of either timing is determined by a 
decision circuit 204. The decision circuit 204 can 
be arranged in the same manner as in Fig. 33 or 
34. 

The color filters of the image pickup element 
need not be R, G, and B filters. As shown in Fig. 
47, a Beyer array having a Y filter (i.e., a filter 
having spectral characteristics close to those of the 
luminance signal), an R filter, and a B filter respec- 
tively corresponding to the first, second, and third 
filters may be used. As shown in Fig. 48, a Beyer 
array having a W (white) filter, an R filter, and a B 
filter respectively corresponding to the first, sec- 
ond, and third filters may be used. Any array can 
be employed if color difference signals R-Y and B- 
Y represented by matrix (1) can be obtained by an 
arithmetic operation using the first and second dif- 
ference signals when the first color signal is sepa- 
rated as shown in Fig. 36 or 42 to form the first 
and second difference signals (the matrix coeffi- 
cients need not be limited to the specific ones 
shown in the matrix for obtaining the signals R-Y 
and B-Y). 

As described above, since the color filters con- 
stituting the Beyer array are used in the image 
pickup element, appropriate signal processing is 
performed. There is therefore provided a color im- 
age pickup apparatus capable of producing an im- 
age having a high resolution, reduced moire, and a 
high S/N ratio. 

The present invention will be described in de- 
tail with reference to still another embodiment 
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hereinafter. 

Fig. 51 is a block diagram showing a "color 
image pickup apparatus" as the eighteenth em- 
bodiment of the present invention. R, G. and B 

5 filers (filter array) having a Beyer array shown in 
Fig. 4A are arranged in an image pickup element 
(sensor) 101. An image signal read from the image 
pickup element 101 pixel by pixel is separated into 
R, G, and B signals by a color separation unit 102. 

10 The R, G, and B signals are subjected to white 
balance adjustment by a white balance unit 1 1 1 on 
the basis of color temperature information obtained 
from a white balance sensor (AWB) 120. An output 
from the white balance unit 111 is subjected to 7 

75 correction by a 7 correction unit 112. An output 
from the 7 correction unit 112 is A/D-converted by 
an A/D (analog-to-digital) converter 103. 

A luminance signal is switched by a switch 
circuit (SWY) 126 and is rearranged in a read 

20 order. The rearranged luminance signal is read as 
a luminance signal Y s including a high-frequency 
component. This luminance signal Y s is added by 
an adder 117 to signals representing the products 
between constants and a first difference signal, i.e., 

25 an (R ( 7 th power) - G1 (7th power)) signal and a 
second difference signal, i.e., a (B (7th power) - G2 
(7th power)) signal. An output from the adder 117 
is D/A-converted by a D/A (digital-to-analog) con- 
verter 118. 

30 Meanwhile, a G (7th power) signal of outputs 

from the A/D converter 103 is separated by a 
switch (SW) 128 into Gi ( 7 th power) and G 2 ( 7 th 
power) signals located at positions shown in Fig. 
52. This operation can be performed by operating 

35 the switch 128, e.g., every horizontal scanning pe- 
riod. The separated Gi (7th power) and G2 (7th 
power) signals are input together with an R (7th 
power) signal and a B (7th power) signal to inter- 
polation filters 106, 107, 108, and 109, respectively, 

40 thereby obtaining coincident R ( 7 th power), G1 (7th 
power), G2 (7th power), and B ( 7 th power) signals. 
In addition to the coincidence operation by inter- 
polation in the interpolation filters 106 to 109, linear 
processing such as two-dimensional low-pass filter- 

45 ing and edge emphasis are performed. Since these 
processing operations are linear processing oper- 
ations, they can be performed after addition and 
matrix processing (both will be described in detail 
later). 

50 The coincident R (7th power) and G1 (7th 

power) signals are subtracted from each other by 
an adder 129, and an (R (7th power) - G1 (7th 
power)) signal is output from the adder 129. Simi- 
larly, the B (7th power) and G2 (7th power) signals 

55 are subtracted from each other by an adder 130, 
and a (B (7th power) - G 2 (7th power)) signal is 
output from the adder 130. These output signals 
are input to a color difference matrix processor 

24 
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113, and the following matrix operation is per- 
formed to derive color difference signals R-Y and 
B-Y. 



'r- 


Y 




' 0 .70-0 . 


11 


B- 


Y 




-0 .30+0 


. 89 



Assume a monochrome object at a position 
(1/2P Hl 0) in the frequency space is picked up by 
the image pickup element 101. This object repre- 
sents vertical fringes having a period 2P H . For this 
object, conditions R (7th power) = Gi (7th power) 
and B (7th power) = G 2 (7th power) are estab- 
lished. The color difference signals, i.e., the (R (7th 
power) - Gi (7th power)) signal and the (B ( 7 th 
power) - G 2 (7th power)) signal respectively output 
from the adders 129 and 130 become zero. There- 
fore, the color difference signals R-Y and B-Y out- 
put from the color difference matrix processor 113 
become zero and are not output. This indicates that 
the carrier of the color difference signal at the 
frequency point (1/2P H ,0) is canceled. In other 
words, the carrier of the R (7th power) signal is set 
in phase with that of the Gi (7th power), and the 
carrier of the B (7th power) is also in phase with 
that of the G2 (7th power) signal at the frequency 
point (1/2P H ,0). Since these difference signals, i.e., 
the (R (7th power) - Gi (7th power)) signal and the 
(B (7th power) - G2 (7th power)) signal, can cancel 
their carriers at this frequency, the carriers of the 
color difference signals are not generated. These 
color difference signals are D/A-con verted by D/A 
converters 114 and 1 1 5. 

The first difference signal (R (7th power) - Gi - 
(7th power)) signal and the second difference sig- 
nal (B (7th power) - G2 (7th power)) are multiplied 
with constants by constant multipliers 132 and 133 
and are added by an adder 134. An output from 
the adder 134 is added to the luminance signal Y s 
by the adder 117, thereby obtaining a luminance 
signal Y whose spectral characteristics are cor- 
rected. 

The principle of this operation will be described 
below. 

Assume that outputs from the respective pixels 
of the image pickup element 101 are simply syn- 
thesized, and the resultant luminance signal is de- 
fined as Y s . The luminance signal Y s is obtained by 
switching the output signals from the respective 
pixels by means of the switch circuit 126. One of 
the color-separated components, e.g., a G signal 
may be used. To the contrary, the luminance signal 
whose spectral characteristics are corrected to be 
equal to visibility sensitivity is defined as Y L . The 
luminance signal Y L can be formed by linear cou- 



pling of coincident color signals and is defined as 
follows: 

Y L = 5R Y + aGi Y + jSG 2 Y + *B y 
5 for 5 + a + j8 + € = 1 (1) 

In the NTSC scheme, 

8 = 0.30, a + 0 = 0.59, € =0.11 (2) 

10 

The luminance signal Y L has a lower band than 
that of the luminance signal Y s . Of all the lu- 
minance signals Y s , a luminance signal having the 
same band as that of the luminance signal Y L is 
75 defined as Y SL . 

At this time, the luminance signal Y is cor- 
rected by substituting the low-frequency compo- 
nent Y SL of the luminance signal Y s with the lu- 
minance signal Y L having the correct spectral char- 
20 acteristics. That is, 

Y = (Y s - Y SL ) + Y L = Y L + (Y L - Y SL ) (3) 

The luminance signal Y SL can be expressed by 
25 linear coupling of respective color signals and is 
defined as follows: 

Y SL = sR Y + tGi 7 + uG 2 Y + wB Y 
for s + t + u + w = 1 (4) 

30 

At this time, terms within the parentheses on 
the right side in equation (3) are defined as follows: 

Y L - Y SL = (5 - s)R Y - (t - a)Gi 7 + (« - w)B Y - (u - 
35 jS)G 2 

wherein if 5 - s = t - a, then s ' + t = a + 5 
and 6+a + £+ t = s + t + u+ w(=1) t so 
that 

40 

j8 + £ = u + w 
therefore 
45 £ - w = u - £ 



If the following conditions are given: 
so Ci = 5 - s = t - a (5) 
C 2 = e - w = u - ff (6) 
the following equation is derived: 

55 

Y L - Y SL = Ci(R* - Gi*) + C 2 (B T - G 2 Y ) 

A substitution of this equation into equation (3) 

25 
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yields the following: 

Y = Y s + Ct(R' - Gi 7 ) + C 2 (R Y - G 2 y ) (7) 

The spectral characteristics of the luminance 
signal Y are obtained by adding products of the 
constants and the first and second difference sig- 
nals (R (>th power) - Gi (yth power)) and (B ( 7 th 
power) - G2 (7th power)) to the simply synthesized 
luminance signal Y s . 

When the luminance signal Y s is obtained by a 
switching operation of the switch circuit 126, values 
s, t, u, and w in equation (4) satisfy the following 
equation: 

s = t = u = w = 0.25 (8) 

the following relations are derived from equations 
(2), (5), and (6): 

a = 0.20, £ = 0.39, 

C1 = 0.05, C 2 = -0.14 (9) 

When the luminance signal Y s is obtained us- 
ing the G signal, the values s, t, u, and w are given 
as follows: ~~ ~ 

s = w = 0, t = u = 0.5 (10) 



The following relations are then derived from 
equations (2), (5), and (6): 

a = 0.20, £ = 0.39, 

Ci = 0.30, C 2 =0.11 (11) 

The luminance signal whose spectral charac- 
teristics are corrected as described above is D/A- 
converted by the D/A converter 118, and an analog 
signal is output from the D/A converter 118. Since 
the color difference signals R-Y and B-Y and the 
low-frequency component Y L of the luminance sig- 
nal generally have bandwidths sufficiently narrower 
than the luminance signal Y, operations of the 
adders 129 and 130, the color difference matrix 
processor 113, and the like for the interpolated 
coincident R (7th power), Gi (7th power), G2 (7th 
power), and B (7th power) signals may be per- 
formed using a clock having a lower clock rate than 
that of the luminance signal Y by extraction or 
interlacing. 

As described above, according to this embodi- 
ment, since stripe filters are not used, the moire 
can be minimized. In addition, since complemen- 
tary color filters are not used, a high S/N ratio can 
be obtained. Since the color signals carriers at the 
frequency point (1/2P Hl 0) can be canceled, a high- 
er resolution can be obtained. The luminance in- 
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formation whose spectral characteristics are cor- 
rected can be obtained by a simple technique. 

The nineteenth embodiment of the present in- 
vention will be described below. 
5 A filter array as in the eighteenth embodiment 

(Fig. 4A) is arranged in an image pickup element 
101. The overall arrangement of the eighteenth 
embodiment is substantially the same as that of 
Fig. 51, except that a G (7th power) signal output 

70 from an A/D converter 103 is separated into Gi - 
(7th power) and G2 (7th power) signals located at 
positions shown in Fig. 53 by a switch 128. 

Assume that a monochrome object at a posi- 
tion (0,1 /2P V ) in the frequency space is picked up 

75 by the image pickup element 101. This object 
represents horizontal fringes having a period 2P V . 
For this object, conditions R (7th power) = Gi (7th 
power) and B (7th power) = G 2 (7th power) are 
established. The color difference signals, i.e., the 

20 (R ( 7 th power) - Gi (7th power)) signal and the (B 
(7th power) - G2 (7th power)) signal respectively 
output from adders 129 and 130 become zero. 
Therefore, the color difference signals R-Y and B-Y 
output from a color difference matrix processor 1 13 

25 become zero and are not output. This indicates that 
the carrier of the color difference signal at the 
frequency point (0,1/2P V ) is canceled. In other 
words, the carrier of the R (7th power) signal is set 
in phase with that of the Gi (7th power), and the 

30 carrier of the B ( 7 th power) is also in phase with 
that of the G2 (7th power) signal at the frequency 
point (0,1 /2P V ). Since these difference signals, i.e., 
the (R (7th power) - Gi (7th power)) signal and the 
(B (7th power) - G 2 (7th power)) signal, can cancel 

35 their carriers at this frequency, the carriers of the 
color difference signals are not generated. The 
luminance information is formed in the same man- 
ner as in the eighteenth embodiment. 

When an output signal obtained from the ap- 

40 paratus shown in Fig. 51 is to be recorded in an 
analog form, D/A converters 118, 114, and 115 are 
necessary. However, when this output signal is 
recorded in a digital form in a magnetic medium, 
an optomagnetic medium, an E 2 PROM (Electrically 

45 Erasable PROM), or the like, the above D/A con- 
verters can be omitted. 

The twentieth embodiment of the present in- 
vention will be described below. 

Fig. 54 is a block diagram showing a signal 

50 processing section of a "color image pickup ap- 
paratus" of the twentieth embodiment of the 
present invention. R, G, and B filters having a 
Beyer array shown in Fig. 4A are arranged in an 
image pickup element (sensor) 401 . An image sig- 

55 nal read from the image pickup element 401 pixel 
by pixel is separated into R, G, and B signals by a 
color separation unit 402. The R, G, and B signals 
are subjected to white balance adjustment by a 

26 
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white balance unit 41 1 on the basis of color tem- 
perature information obtained from a white balance 
sensor (AWB) 420. An output from the white bal- 
ance unit 41 1 is subjected to 7 correction by a 7 
correction unit 412. An output from the 7 correction 5 
unit 412 is A/D-converted by an A/D (analog-to- 
digital) converter 403. 

A luminance signal is obtained as follows. An 
offset sampling structure of a G (7th power) signal 
is two-dimensionally interpolated by an interpola- 10 
tion filter 425, the spectral characteristics of the G 
signal are corrected by first and second difference 
signals (to be described later) by an adder 417, 
and an output from the interpolation filter 425 is 
D/A-converted by a D/A converter 418. An analog 75 
signal is thus output from the D/A converter 418. In 
addition to the coincidence operation by interpola- 
tion in the interpolation filter 425, processing such 
as two-dimensional low-pass filtering and edge em- 
phasis are performed. 20 

Meanwhile, a G (7th power) signal of outputs 
from the A/D converter 403 is separated by a 
switch 428 into G1 (7th power) and G2 (7th power) 
signals located at positions shown in Fig. 52. This 
operation can be performed by operating the 25 
switch 428, e.g., every horizontal scanning period. 
The separated Gi (7th power) and G2 (7th power) 
signals are input together with an R (7th power) 
signal and a B (7th power) signal to interpolation 
filters 406, 407, 408, and 409, respectively, thereby 30 
obtaining coincident R (7th power), G1 (7th power), 
G2 (7th power), and B ( 7 th power) signals. In 
addition to the coincidence operation by interpola- 
tion in the interpolation filters 406 to 409, linear 
processing such as two-dimensional low-pass filter- 35 
ing and edge emphasis are performed. Since these 
processing operations are linear processing oper- 
ations, they can be performed after addition (to be 
described in detail later). 

The coincident R (7th power) and Gi (7th 40 
power) signals are subtracted from each other by 
an adder 429, and a first difference signal, i.e., an 
(R (7th power) - Gi (7th power)) signal is output 
from the adder 429. Similarly, the B ( 7 th power) 
and G2 (7th power) signals are subtracted from as 
each other by an adder 430, and a second dif- 
ference signal, i.e., a (B (7th power) - G2 (7th 
power)) signal is output from the adder 430. Color 
difference signals R-Y and B-Y are generated by a 
color difference matrix processor 413. 50 

Assume that a monochrome object at a posi- 
tion (1/2P Hl 0) in the frequency space is picked up 
by the image pickup element 401. This object 
represents vertical fringes having a period 2P H . For 
this object, the (R (7th power) - Gi ( 7 th power)) 55 
signal and the (B ( 7 th power) - G2 (7th power)) 
signal respectively output from the adders 429 and 
430 become zero and are not output. This indicates 



that the carrier of the color difference signal at the 
frequency point (1/2P H ,0) is canceled. 

The luminance signal is formed by directly 
using the G signal as Y s in the description of the 
principle of luminance signal correction. 

The twenty-first embodiment of the present in- 
vention will be described below. 

An image pickup element has a Beyer array 
(Fig. 4A) as in the twentieth embodiment. The 
overall arrangement of the twentieth embodiment is 
substantially the same as that of the arrangement 
shown in Fig. 54, except that a Y (7th power) signal 
output from an A/D converter 403 is separated into 
Gi (7th power) and G2 (7th power) signals located 
at positions shown in Fig, 53 by a switch 428. 

Assume that a monochrome object at a posi- 
tion (0,1 /2P V ) in the frequency space is picked up 
by an image pickup element 401. This object re- 
presents horizontal fringes having a period 2P V . For 
this object, the (R (7th power) - Gi (7th power)) 
signal and the (B (7th power) - G2 (7th power)) 
signal respectively output from adders 429 and 430 
become zero and are not output. This indicates that 
the carrier of the color difference signal at the 
frequency point (0,1/2P V ) is canceled. The lumi- 
nance signal is formed in the same manner as in 
the twentieth embodiment. 

The color filters of the image pickup element 
need not be R, G, and B filters. As shown in Fig. 
47, a Beyer array having a Y filter (i.e., a filter 
having spectral characteristics close to those of the 
luminance signal), an R filter, and a B filter respec- 
tively corresponding to the first, second, and third 
filters may be used. As shown in Fig. 48, a Beyer 
array having a W (white) filter, an R filter, and a B 
filter respectively corresponding to the first, sec- 
ond, and third filters may be used. Any array can 
be employed if color difference signals R-Y and B- 
Y represented by matrix (1) can be obtained by an 
arithmetic operation using the first and second dif- 
ference signals when the first color signal is sepa- 
rated as shown in Fig. 52 or 53 to form the first 
and second difference signals (the matrix coeffi- 
cients need not be limited to the specific ones 
shown in matrix (1) for causing the color difference 
matrix processor 113 to obtain the color difference 
signals R-Y and B-Y). 

As described above, there is provided a color 
image pickup apparatus capable of obtaining lu- 
minance information whose spectral characteristics 
are corrected by a simple technique and of obtain- 
ing a good image having a high resolution, reduced 
moire and a high S/N ratio. 

The present invention will be described in de- 
tail with reference to still another embodiment 
hereinafter. 

Fig. 55 is a block diagram showing a "color 
image pickup apparatus" as the twenty-second em- 
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bodiment of the present invention. R, G, and B 
filters (filter array) having a Beyer array shown in 
Fig. 4A are arranged in an image pickup element 
(sensor) 101. An image signal read from the image 
pickup element 101 pixel by pixel is separated into 
R, G, and B signals by a color separation unit 102, 
The R, G, and B signals are subjected to white 
balance adjustment by a white balance unit 111 on 
the basis of color temperature information obtained 
from a white balance sensor (AWB) 120. An output 
from the white balance unit 111 is subjected to y 
correction by a y correction unit 112. An output 
from the y correction unit 112 is A/D-converted by 
an A/D (analog-to-digital) converter 103. 

A luminance signal is switched by a switch 
circuit (SWY) 126 and is rearranged in a read 
order. The rearranged luminance signal is extracted 
as a luminance signal Y s . The luminance signal Y s 
is added by an adder 117 to products between 
constants and a first or third difference signal (R 
(7th power) - Gi (7th power)) and a second or 
fourth difference signal (B (7th power) - G2 (7th 
power)) (these signals will be described in detail 
later). Sum signals are converted by a D/A (digital- 
to-analog) converter 118, and analog signals are 
output from the D/A converter 118. 

Meanwhile, a G (7th power) signal of outputs 
from the A/D converter 103 is separated by a 
switch (SW) 128 into Gr (7th power) and G 2 (7th 
power) signals located at the illustrated positions at 
two timings, i.e., timing 1 shown in Fig. 56A and 
timing 2 shown in Fig. 56B. This operation can be 
performed by operating the switch 128, e.g., every 
horizontal scanning period. Switching between tim- 
ings 1 and 2 is performed by a decision circuit 131 
(to be described later) in accordance with a lu- 
minance signal of an object to be photographed. 
The separated G1 (7th power) and G2 (7th power) 
signals are input together with an R (7th power) 
signal and a B (7th power) signal to interpolation 
filters 106, 107, 108, and 109, respectively, thereby 
obtaining coincident R (7th power), G1 (7th power), 
G2 (7th power), and B (7th power) signals. In 
addition to the coincidence operation by interpola- 
tion in the interpolation filters 106 to 109, linear 
processing such as two-dimensional low-pass filter- 
ing and edge emphasis are performed. Since these 
processing operations are linear processing oper- 
ations, they can be performed after addition and 
matrix processing (both will be described in detail 
later). 

The coincident R (7th power) and Gi (7th 
power) signals are subtracted from each other by 
an adder 129, and the first and third difference 
signals (R (7th power) - Gi (7th power)) are output 
from the adder 129. Similarly, the B (7th power) 
and G2 (7th power) signals are subtracted from 
each other by an adder 130, and the second and 
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fourth signals (B ( 7 th power) - G2 (7th power)) are 
output from the adder 130. These output signals 
are input to a color difference matrix processor 
113, and the following matrix operation is per- 
5 formed to derive color difference signals R-Y and 
B-Y. 



'r- 


Y 




0.70-0. 


11 


B- 






-0 .30+0 


•89 



R y -Gi 



Assume that switching between the Gi (7th 

75 power) and G2 (7th power) signals is performed at 
timing 1 shown in Fig. 56A, and that a monoch- 
rome object at a position (1/2P H ,0) in the frequency 
space is picked up by the image pickup element 
101. This object represents vertical fringes having 

20 a period 2P H - For this object, conditions R (7th 
power) = Gi (7th power) and B (7th power) = G2 
( 7 th power) are established. The difference signals 
(R ( 7 th power) - Gi (7th power)) signal and (B ( 7 th 
power) - G 2 (7th power)) respectively output from 

25 the adders 129 and 130 become zero. Therefore, 
the color difference signals R-Y and B-Y output 
from the color difference matrix processor 113 be- 
come zero and are not output. This indicates that 
the carrier of the color difference signal at the 

30 frequency point (1/2P H ,0) is canceled. In other 
words, the carrier of the R (7th power) signal is set 
in phase with that of the Gi (7th power), and the 
carrier of the B ( 7 th power) is also in phase with 
that of the G2 (7th power) signal at the frequency 

35 point (1/2P H ,0). Since the first and second dif- 
ference signals (R (7th power) - Gi (7th power)) 
and (B (7th power) - G2 (7th power)) can cancel 
their carriers at this frequency, the carriers of the 
color difference signals are not generated. 

40 Assume that switching between the Gi (7th 

power) and G2 (7th power) signals is performed at 
timing 2 shown in Fig. 56B, and that a monoch- 
rome object at a position (0,1 /2P V ) in the frequency 
space is picked up by the image pickup element 

45 101. This object represents horizontal fringes hav- 
ing a period 2P V . For this object, the difference 
signals (R ( 7 th power) - Gi ( 7 th power)) and (B (7th 
power) - G2 (7th power)) respectively output from 
the adders 129 and 130 become zero. Therefore, 

50 the color difference signals R-Y and B-Y output 
from the color difference matrix processor 113 be- 
come zero and are not output. This indicates that 
the carrier of the color difference signal at the 
frequency point (0,1 /2P V ) is canceled. In other 

55 words, the carrier of the R (7th power) signal is set 
in phase with that of the Gi (7th power), and the 
carrier of the B ( 7 th power) is also in phase with 
that of the G 2 (7th power) signal at the frequency 
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point (0,1/2P V ). Since the third and fourth difference 
signals (R ( 7 th power) - Gi (7th power)) and (B ( 7 th 
power) - G2 (7th power)) can cancel their carriers 
at this frequency, the carriers of the color dif- 
ference signals are not generated. 

These color difference signals are converted 
into analog signals by the D/A converters 114 and 
115, and the analog signals are output. 

The third difference signal (R (7th power) - G1 
(yth power)) and the fourth difference signal (B ( 7 th 
power) - G2 (7th power)) are multiplied with con- 
stants by constant multipliers 132 and 133 and are 
added to each other by an adder 134. An output 
from the adder 134 is added to the luminance 
signal Y s by the adder 117, thereby obtaining the 
luminance signal Y whose spectral characteristics 
are corrected. 

The principle of the above operation has been 
described previously, and a description thereof will 
be omitted. 

When the luminance signal Y s is obtained by a 
switching operation of the switch circuit 126, values 
s, t, u, and w in equation (4) satisfy the following 
equation: ~ 

s = t = u = w = 0.25 (8) 

the following relations are derived from equations 
(2), (5), and (6): 

a = 0.20, & = 0.39, 

C1 = 0.05, C 2 = -0.14 (9) 

When the luminance signal Y s is obtained us- 
ing the G signal, the values s, t, u, and w are given 
as follows: ~ ~~ 

s = w = 0, t = u = 0.5 (10) 

The following relations are then derived from 
equations (2), (5), and (6): 

a = 0.20, £ = 0.39, 

C, = 0.30, 0 2 = 0.11 (11) 

The luminance signal whose spectral charac- 
teristics are corrected as described above is D/A- 
converted by the D/A converter 1 18, and an analog 
signal is output from the D/A converter 118. Since 
the color difference signals R-Y and B-Y and the 
low-frequency component Y L of the luminance sig- 
nal generally have bandwidths sufficiently narrower 
than the luminance signal Y, operations of the 
adders 129 and 130, the color difference matrix 
processor 113, the adder 117 and the like for the 
interpolated coincident R (7th power), G1 (7th pow- 
er), G2 (7th power), and B (7th power) signals may 
be performed using a clock having a lower clock 
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rate than that of the luminance signal Y by extrac- 
tion or interlacing. 

When an output signal obtained from the ap- 
paratus shown in Fig. 55 is to be recorded in an 
5 analog form, D/A converters 118, 114, and 115 are 
necessary. However, when this output signal is 
recorded in a digital form in a magnetic medium, 
an optomagnetic medium, an E 2 PROM (Electrically 
Erasable PROM), or the like, the above D/A con- 
10 verters can be omitted. 

A decision circuit 131 can be arranged in the 
same manner as in Fig. 33 or 34, and a repetitive 
description will be omitted for descriptive simplic- 
ity. 

15 As described above, according to this embodi- 

ment, since the color filters constituting the Beyer 
array are used, the moire can be minimized, and a 
high S/N ratio can be obtained. In addition, since 
the signal processing means suitable for the Beyer 

20 array is used, a high resolution can be obtained. 

Luminance information whose spectral charac- 
teristics are corrected can be obtained by a simple 
technique. 

The twenty-third embodiment of the present 

25 invention will be described below. 

Fig. 57 is a block diagram showing a signal 
processing section of a "color image pickup ap- 
paratus" of the twenty-third embodiment of the 
present invention. R, G, and B filters having a 

30 Beyer array shown in Fig. 4A are arranged in an 
image pickup element (sensor) 401 . An image sig- 
nal read from the image pickup element 401 pixel 
by pixel is separated into R, G, and B signals by a 
color separation unit 402. The R, G, and B signals 

35 are subjected to white balance adjustment by a 
white balance unit 41 1 on the basis of color tem- 
perature information obtained from a white balance 
sensor (AWB) 420. An output from the white bal- 
ance unit 41 1 is subjected to 7 correction by a 7 

40 correction unit 412. An output from the 7 correction 
unit 412 is A/D-converted by an A/D (analog-to- 
digital) converter 403. 

A luminance signal is obtained as follows. An 
offset sampling structure of a G (7th power) signal 

45 is two-dimensionally interpolated by an interpola- 
tion filter 425, and an output from the interpolation 
filter 425 is D/A-converted by a D/A converter 418. 
An analog signal is thus output from the D/A con- 
verter 418. In addition to the coincidence operation 

50 by interpolation in the interpolation filter 425, pro- 
cessing such as two-dimensional low-pass filtering 
and edge emphasis are performed. 

Meanwhile, a G (7th power) signal of outputs 
from the A/D converter 403 is separated by a 

55 switch 428 into Gi ( 7 th power) and G2 (7th power) 
signals located at the illustrated positions at two 
timings, i.e., timing 1 shown in Fig. 56A and timing 
2 shown in Fig. 56B. This operation can be per- 
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formed by operating the switch 428, e.g., every 
horizontal scanning period. Switching between tim- 
ings 1 and 2 is performed by a decision circuit 431 
(to be described later) in accordance with a lu- 
minance signal of an object to be photographed. 5 

The separated Gi ( 7 th power) and G2 (7th 
power) signals are input together with an R ( 7 th 
power) signal and a B (7th power) signal to inter- 
polation filters 406, 407, 408, and 409, respectively, 
thereby obtaining coincident R ( 7 th power), G1 ( 7 th 10 
power), G2 ( 7 th power), and B ( 7 th power) signals. 
In addition to the coincidence operation by inter- 
polation in the interpolation filters 406 to 409, linear 
processing such as two-dimensional low-pass filter- 
ing and edge emphasis are performed. Since these 75 
processing operations are linear processing oper- 
ations, they can be performed after addition (to be 
described in detail later). The coincident R ( 7 th 
power) and Gi ( 7 th power) signals are subtracted 
from each other by an adder 429, and difference 20 
signals (R ( 7 th power) - Gi ( 7 th power)) are output 
from the adder 429. Similarly, the B ( 7 th power) 
and G2 ( 7 th power) signals are subtracted from 
each other by an adder 430, and difference signals 
(B ( 7 th power) - G2 ( 7 th power)) are output from 25 
the adder 430. Color difference signals are gen- 
erated by a color difference matrix processor 413 
as in the first embodiment. 

Assume that switching between the Gi (7th 
power) and G2 (7th power) signals is performed at 30 
timing 1 shown in Fig. 56A, and that a monoch- 
rome object at a position (1/2P H ,0) in the frequency 
space is picked up by the image pickup element 
401. This object represents vertical fringes having 
a period 2P H . For this object, the difference signals 35 
(R (7th power) - Gi (7th power)) and (B ( 7 th power) 
- G2 (7th power)) respectively output from the 
adders 429 and 430 become zero and are not 
output. This indicates that the carrier of the color 
difference signal at the frequency point (1/2P H ,0) is 40 
canceled. In other words, the carrier of the R (7th 
power) signal is set in phase with that of the Gi - 
( 7 th power), and the carrier of the B (7th power) is 
also in phase with that of the G2 (7th power) signal 
at the frequency point (1/2P H ,0). Since the first and 45 
second difference signals (R (7th power) - Gi (7th 
power)) and (B ( 7 th power) - G2 (7th power)) can 
cancel their carriers at this frequency, the carriers 
of the color difference signals are not generated. 

Assume that switching between the Gi (7th so 
power) and G2 (7th power) signals is performed at 
timing 2 shown in Fig. 56B, and that a monoch- 
rome object at a position (0,1 /2P V ) in the frequency 
space is picked up by the image pickup element 
401. This object represents horizontal fringes hav- 55 
ing a period 2P V . For this object, the difference 
signals (R (7th power) - Gi (7th power)) and (B ( 7 th 
power) - G2 (7th power)) respectively output from 



the adders 429 and 430 become zero. Therefore, 
the color difference signals R-Y and B-Y output 
from the adders 429 and 430 become zero and are 
not output. This indicates that the carrier of the 
color difference signal at the frequency point 
(0,1/2P V ) is canceled. In other words, the carrier of 
the R (7th power) signal is set in phase with that of 
the Gi (7th power), and the carrier of the B ( 7 th 
power) is also in phase with that of the G2 (7th 
power) signal at the frequency point (0,1 /2P V ). 
Since the third and fourth difference signals (R (7th 
power) - Gi (7th power)) and (B (7th power) - G2 - 
(7th power)) can cancel their carriers at this fre- 
quency, the carriers of the color difference signals 
are not generated. These color difference signals 
are converted into analog signals by the D/A con- 
verters 414 and 415, and the analog signals are 
output. Since the color difference signals R-Y and 
B-Y generally have bandwidths sufficiently narrow- 
er than the luminance signal Y, operations of the 
adders 429 and 430, and the like for the interpo- 
lated coincident R (7th power), Gi (7th power), G2 
(7th power), and B (7th power) signals may be 
performed using a clock having a lower clock rate 
than that of the luminance signal Y by extraction or 
interlacing. 

When an output signal obtained from the pro- 
cessing block shown in Fig. 57 is to be recorded in 
an analog form, the D/A converters 418, 414, and 
415 are necessary. However, when this output sig- 
nal is recorded in a digital form in a magnetic 
medium, an optomagnetic medium, an E 2 PROM 
(Electrically Erasable PROM), or the like, the above 
D/A converters can be omitted. 

The color filters of the image pickup element 
need not be R, G, and B filters. As shown in Fig. 
47, a Beyer array having a Y filter (i.e., a filter 
having spectral characteristics close to those of the 
luminance signal), an R filter, and a B filter respec- 
tively corresponding to the first, second, and third 
filters may be used. As shown in Fig. 48, a Beyer 
array having a W (white) filter, an R filter, and a B 
filter respectively corresponding to the first, sec- 
ond, and third filters may be used. Any array can 
be employed if color difference signals R-Y and B- 
Y represented by matrix (1 ) can be obtained by an 
arithmetic operation using the first to fourth dif- 
ference signals when the first color signal is sepa- 
rated as shown in Fig. 56A or 56B to form the first 
to fourth difference signals (the matrix coefficients 
need not be limited to the specific ones shown in 
matrix (1) for causing the color difference matrix 
processor 1 13 to obtain the color difference signals 
R-Y and B-Y). 

As has been described above, according to the 
present invention, since the filter array as the 
Beyer array is arranged in the image pickup ele- 
ment and appropriate signal processing is per- 
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formed, there is provided a color image pickup 
apparatus capable of obtaining luminance informa- 
tion whose spectral characteristics are corrected by 
a simple technique and of obtaining a good image 
having a high resolution, reduced moire and a high 
S/N ratio. 

A color image pickup apparatus for converting 
an object image into an electrical signal having 
luminance information and color information in- 
cludes 

an image pickup element arranged in a rectan- 
gular matrix having pixels arranged at a horizontal 
pitch P H and a vertical pitch P v ; 

a color filter array having first color filters ar- 
ranged in correspondence with the pixels and hav- 
ing an offset sampling structure having a horizontal 
pitch 2P H and a vertical pitch P v and offset by P H 
in a horizontal direction, and second and third color 
filters each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and a verti- 
cal pitch 2P V ; and 

a color information forming circuit having a first 
color signal forming circuit for forming the color 
information in accordance with a difference signal 
between a second color signal and a coincidence 
signal obtained by causing only a first color signal 
which is output from a pixel belonging to the same 
column as that of the second color signal to co- 
incide therewith and a difference signal between a 
third color signal and a coincidence signal obtained 
by causing only a first color signal which is output 
from a pixel belonging to the same column as that 
of the third color signal to coincide therewith, and a 
second color signal forming circuit for forming the 
color information in accordance with a difference 
signal between a second color signal and a co- 
incidence signal obtained by causing only a first 
color signal which is output from a pixel belonging 
to the same row as that of the second color signal 
to coincide therewith and a difference signal be- 
tween a third color signal and a coincidence signal 
obtained by causing only a first color signal which 
is output from a pixel belonging to the same row as 
that of the third color signal to coincide therewith, 
the first and second color signal forming circuits 
operating on the basis of the first, second, and 
third color signals output from pixels corresponding 
to the first, second, and third color filters, and the 
color information forming means switching between 
the first and second color signal forming circuits in 
accordance with a frequency component in a scan- 
ning direction of the object image or a direction 
perpendicular to the scanning direction. 

Claims 

1. A color image pickup apparatus for converting 
an object image into an electrical signal having 



luminance information and color information, 
comprising: 

a. an image pickup element arranged in a 
rectangular matrix having pixels arranged at 

5 a horizontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels 
and having an offset sampling structure hav- 
ing a horizontal pitch 2P H and a vertical 

w pitch P v and offset by P H in a horizontal 

direction, and second and third color filters 
each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and 
a vertical pitch 2P V ; and 

15 c. color information forming means having 

first color signal forming means for forming 
the color information in accordance with a 
difference signal between a second color 
signal and a coincidence signal obtained by 

20 causing only a first color signal which is 

output from a pixel belonging to the same 
column as that of the second color signal to 
coincide therewith and a difference signal 
between a third color signal and a coinci- 

25 dence signal obtained by causing only a 

first color signal which is output from a pixel 
belonging to the same column as that of the 
third color signal to coincide therewith, and 
second color signal forming means for for- 

30 ming the color information in accordance 

with a difference signal between a second 
color signal and a coincidence signal ob- 
tained by causing only a first color signal 
which is output from a pixel belonging to 

35 the same row as that of the second color 

signal to coincide therewith and a difference 
signal between a third color signal and a 
coincidence signal obtained by causing only 
a first color signal which is output from a 

40 pixel belonging to the same row as that of 

the third color signal to coincide therewith, 
said first and second color signal forming 
means operating on the basis of the first, 
second, and third color signals output from 

45 pixels corresponding to the first, second, 

and third color filters, and said color in- 
formation forming means switching between 
said first and second color signal forming 
means in accordance with a frequency com- 

50 ponent in a scanning direction of the object 

image or a direction perpendicular to the 
scanning direction. 

2. A color image pickup apparatus for converting 
55 an object image into an electrical signal having 

luminance information and color information, 
comprising: 

a. an image pickup element arranged in a 

31 
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rectangular matrix having pixels arranged at 
a horizontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels 

and having an offset sampling structure hav- 5 
ing a horizontal pitch 2P H and a vertical 
pitch P v and offset by P H in a horizontal 
direction, and second and third color filters 
each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and 10 
a vertical pitch 2P V ; and 

c. color information forming means for for- 
ming the color information in accordance 
with a difference signal between a second 
color signal and a coincidence signal ob- is 
tained by causing only a first color signal 
which is output from a pixel belonging to 

the same column or row as that of the 
second color signal to coincide therewith 
and a difference signal between a third col- 20 
or signal and a coincidence signal obtained 
by causing only a first color signal which is 
output from a pixel belonging to the same 
column or row as that of the third color 
signal to coincide therewith on the basis of 25 
the first, second, and third color signals 
output from pixels corresponding to the first, 
second, and third color filters. 

A color image pickup apparatus for converting 30 
an object image into an electrical signal having 
luminance information and color information, 
comprising: 

a. an image pickup element arranged in a 
rectangular matrix having pixels arranged at 35 
a horizontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels 
and having an offset sampling structure hav- 
ing a horizontal pitch 2P H and a vertical 40 
pitch P v and offset by P H in a horizontal 
direction, and second and third color filters 
each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and 

a vertical pitch 2P V ; 45 

c. an optical low-pass filter having an optical 
member for splitting an incident beam into 
two beams spaced apart from each other by 
a distance D in a direction forming an angle 

0 with respect to a scanning direction of the 50 
image pickup element in clockwise and 
counterclockwise directions, said optical 
low-pass filter being arranged in an image 
pickup optical system and satisfying the 
following condition: 55 

0.8P H Pv/|PHSine + P v cose| ^ D ^ 
1 .2P H P v /|PHSin0 + P v cos0| 



for 0 £ e £ tt/2; and 



d. color information forming means for for- 
ming the color information in accordance 
with a difference signal between a second 
. color signal and a coincidence signal ob- 
tained by causing only a first color signal 
which is output from a pixel belonging to 
the same column or row as that of the 
second color signal to coincide therewith 
and a difference signal between a third col- 
or signal and a coincidence signal obtained 
by causing only a first color signal which is 
output from a pixel belonging to the same 
column or row as that of the third color 
signal to coincide therewith on the basis of 
the first, second, and third color signals 
output from pixels corresponding to the first, 
second, and third color filters. 

4. A color image pickup apparatus for converting 
an object image into an electrical signal having 
luminance information and color information, 
comprising: 

a. an image pickup element arranged in a 
rectangular matrix having pixels arranged at 
a horizontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels 
and having an offset sampling structure hav- 
ing a horizontal pitch 2P H and a vertical 
pitch P v and offset by P H in a horizontal 
direction, and second and third color filters 
each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and 
a vertical pitch 2P V ; 

c. an optical low-pass filter having an optical 
member for splitting an incident beam into 
two beams spaced apart from each other by 
a distance D in a direction forming an angle 
6 with respect to a scanning direction of the 
image pickup element in clockwise and 
counterclockwise directions, said optical 
low-pass filter being arranged in an image 
pickup optical system and satisfying the 
following condition: 

0.8P H P v /|PHSine + P v cose| ^ D ^ 
1 .2P H P v /|PHSine + Pvcosflj 
for 0 S e £ tt/2; and 



d. color information forming means having 
first color signal forming means for forming 
the color information in accordance with a 
difference signal between a second color 
signal and a coincidence signal obtained by 
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causing only a first color signal which is 
output from a pixel belonging to the same 
column as that of the second color signal to 
coincide therewith and a difference signal 
between a third color signal and a coinci- 5 
dence signal obtained by causing only a 
first color signal which is output from a pixel 
belonging to the same column as that of the 
third color signal to coincide therewith, and 
second color signal forming means for for- w 
ming the color information in accordance 
with a difference signal between a second 
color signal and a coincidence signal ob- 
tained by causing only a first color signal 
which is output from a pixel belonging to 75 
the same row as that of the second color 
signal to coincide therewith and a difference 
signal between a third color signal and a 
coincidence signal obtained by causing only 
a first color signal which is output from a 20 
pixel belonging to the same row as that of 
the third color signal to coincide therewith, 
said first and second color signal forming 
means operating on the basis of the first, 
second, and third color signals output from 25 
pixels corresponding to the first, second, 
and third color filters, and said color in- 
formation forming means switching between 
said first and second color signal forming 
means in accordance with a frequency com- 30 
ponent in a scanning direction of the object 
image or a direction perpendicular to the 
scanning direction. 

A color image pickup apparatus for converting 35 
an object image into an electrical signal having 
luminance information and color information, 
comprising: 

a. an image pickup element arranged in a 
rectangular matrix having pixels arranged at 40 
a horizontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels 
and having an offset sampling structure hav- 
ing a horizontal pitch 2P H and a vertical 45 
pitch P v and offset by P H in a horizontal 
direction, and second and third color filters 
each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and 

a vertical pitch 2P V ; 50 

c. an optical low-pass filter having an optical 
member for splitting an incident beam into 
two beams spaced apart from each other by 
a distance D in a direction forming an angle 

6 with respect to a scanning direction of the 55 
image pickup element in clockwise and 
counterclockwise directions, said optical 
low-pass filter being arranged in an image 



pickup optical system and satisfying the 
following condition: 

O.8P H Pv/|PHSin0 + P v cosfl| < D < 
1 .2P H P v /|PHSine + P v cos0| 
for 0 £ 9 £ tt/2; 

d. color information forming means for for- 
ming the color information in accordance 
with a first difference signal between a sec- 
ond color signal and a coincidence signal 
obtained by causing only a first color signal 
which is output from a pixel belonging to 
the same column or row as that of the 
second color signal to coincide therewith 
and a second difference signal between a 
third color signal and a coincidence signal 
obtained by causing only a first color signal 
which is output from a pixel belonging to 
the same column or row as that of the third 
color signal to coincide therewith on the 
basis of the first, second, and third color 
signals output from pixels corresponding to 
the first, second, and third color filters; and 

e. luminance information forming means for 
forming the luminance information by mul- 
tiplying the first and second difference sig- 
nals with constants, respectively, and add- 
ing the multiplied signals to a signal ob- 
tained by synthesizing at least one of the 
first, second, and third color signals. 

6. A color image pickup apparatus for converting 
an object image into an electrical signal having 
luminance information and color information, 
comprising: 

a. an image pickup element arranged in a 
rectangular matrix having pixels arranged at 
a horizontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels 
and having an offset sampling structure hav- 
ing a horizontal pitch 2P H and a vertical 
pitch P v and offset by P H in a horizontal 
direction, and second and third color filters 
each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and 
a vertical pitch 2P V ; 

c. an optical low-pass filter having an optical 
member for splitting an incident beam into 
two beams spaced apart from each other by 
a distance D in a direction forming an angle 
6 with respect to a scanning direction of the 
image pickup element in clockwise and 
counterclockwise directions, said optical 
low-pass filter being arranged in an image 
pickup optical system and satisfying the 
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following condition: 

O.8P H Pv/|PHSin0 + P v cos0| £ D < 
1 .2P H P v /|PHSin0 + P v cos0| 

for 0 £ e £ tt/2; 5 



d. color information forming means having 
first color signal forming means for forming 

the color information in accordance with a w 
first difference signal between a second col- 
or signal and a coincidence signal obtained 
by causing only a first color signal which is 
output from a pixel belonging to the same 
column as that of the second color signal to 15 
coincide therewith and a second difference 
signal between a third color signal and a 
coincidence signal obtained by causing only 
a first color signal which is output from a 
pixel belonging to the same column as that 20 
of the third color signal to coincide there- 
with, and second color signal forming 
means for forming the color information in 
accordance with a third difference signal 
between a second color signal and a co- 25 
incidence signal obtained by causing only a 
first color signal which is output from a pixel 
belonging to the same row as that of the 
second color signal to coincide therewith 
and a fourth difference signal between a 30 
third color signal and a coincidence signal 
obtained by causing only a first color signal 
which is output from a pixel belonging to 
the same row as that of the third color 
signal to coincide therewith, said first and 35 
second color signal forming means operat- 
ing on the basis of the first, second, and 
third color signals output from pixels cor- 
responding to the first, second, and third 
color filters, and said color information for- 40 
ming means switching between said first 
and second color signal forming means in 
accordance with a frequency component in 
a scanning direction of the object image or 
a direction perpendicular to the scanning 45 
direction; and 

e. luminance information forming means for 
forming the luminance information by mul- 
tiplying the first or third difference signal 

and the second or fourth difference signal so 
with constants, respectively, and adding the 
multiplied signals to a signal obtained by 
synthesizing at least one of the first, sec- 
ond, and third color signals. 
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A color image pickup apparatus for converting 
an object image into an electrical signal having 
luminance information and color information, 



comprising: 

a. an image pickup element arranged in a 
rectangular matrix having pixels arranged at 
a horizontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels 
and having an offset sampling structure hav- 
ing a horizontal pitch 2P H and a vertical 
pitch P v and offset by P H in a horizontal 
direction, and second and third color filters 
each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and 
a vertical pitch 2P V ; 

c. color information forming means for for- 
ming the color information in accordance 
with a first difference signal between a sec- 
ond color signal and a coincidence signal 
obtained by causing only a first color signal 
which is output from a pixel belonging to 
the same column or row as that of the 
second color signal to coincide therewith 
and a second difference signal between a 
third color signal and a coincidence signal 
obtained by causing only a first color signal 
which is output from a pixel belonging to 
the same column or row as that of the third 
color signal to coincide therewith on the 
basis of the first, second, and third color 
signals output from pixels corresponding to 
the first, second, and third color filters; and 

d. luminance information forming means for 
forming the luminance information by mul- 
tiplying the first and second difference sig- 
nals with constants, respectively, and add- 
ing the multiplied signals to a signal ob- 
tained by synthesizing at least one of the 
first, second, and third color signals. 

8. A color image pickup apparatus for converting 
an object image into an electrical signal having 
luminance information and color information, 
comprising: 

a. an image pickup element arranged in a 
rectangular matrix having pixels arranged at 
a horizontal pitch P H and a vertical pitch P v ; 

b. a color filter array having first color filters 
arranged in correspondence with the pixels 
and having an offset sampling structure hav- 
ing a horizontal pitch 2P H and a vertical 
pitch P v and offset by P H in a horizontal 
direction, and second and third color filters 
each having a rectangular matrix sampling 
structure having a horizontal pitch 2P H and 
a vertical pitch 2P V ; 

c. color information forming means having 
first color signal forming means for forming 
the color information in accordance with a 
first difference signal between a second col- 
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or signal and a coincidence signal obtained 
by causing only a first color signal which is 
output from a pixel belonging to the same 
column as that of the second color signal to 
coincide therewith and a second difference 5 
signal between a third color signal and a 
coincidence signal obtained by causing only 
a first color signal which is output from a 
pixel belonging to the same column as that 
of the third color signal to coincide there- w 
with, and second color signal forming 
means for forming the color information in 
accordance with a third difference signal 
between a second color signal and a co- 
incidence signal obtained by causing only a 15 
first color signal which is output from a pixel 
belonging to the same row as that of the 
second color signal to coincide therewith 
and a fourth difference signal between a 
third color signal and a coincidence signal 20 
obtained by causing only a first color signal 
which is output from a pixel belonging to 
the same row as that of the third color 
signal to coincide therewith, said first and 
second color signal forming means operat- 25 
ing on the basis of the first, second, and 
third color signals output from pixels cor- 
responding to the first, second, and third 
color filters, and said color information for- 
ming means switching between said first 30 
and second color signal forming means in 
accordance with a frequency component in 
a scanning direction of the object image or 
a direction perpendicular to the scanning 
direction; and 35 
d. luminance information forming means for 
forming the luminance information by mul- 
tiplying the first or third difference signal 
and the second or fourth difference signal 
with constants, respectively, and adding the 40 
multiplied signals to a signal obtained by 
synthesizing at least one of the first, sec- 
ond, and third color signals. 
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FIG. 56 A 
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© A color image pickup apparatus for converting 
an object image into an electrical signal having lu- 
minance information and color information includes 

an image pickup element arranged in a rectan- 
gular matrix having pixels arranged at a horizontal 
pitch P H and a vertical pitch P v ; 

a color filter array having first color filters ar- 
ranged in correspondence with the pixels and having 
an offset sampling structure having a horizontal pitch 
2P H and a vertical pitch P v and offset by P H in a 
horizontal direction, and second and third color filters 
each having a rectangular matrix sampling structure 
having a horizontal pitch 2P H and a vertical pitch 
2P V ; and 

a color information forming circuit having a first 
color signal forming circuit for forming the color 
information in accordance with a difference signal 
between a second color signal and a coincidence 
signal obtained by causing only a first color signal 
which is output from a pixel belonging to the same 
column as that of the second color signal to coincide 
therewith and a difference signal between a third 
color signal and a coincidence signal obtained by 



causing only a first color signal which is output from 
a pixel belonging to the same column as that of the 
third color signal to coincide therewith, and a second 
color signal forming circuit for forming the color 
information in accordance with a difference signal 
between a second color signal and a coincidence 
signal obtained by causing only a first color signal 
which is output, from a pixel belonging to the same 
row as that of the second color signal to coincide 
therewith and a difference signal between a third 
color signal and a coincidence signal obtained by 
causing only a first color signal which is output from 
a pixel belonging to the same row as that of the third 
color signal to coincide therewith, the first and sec- 
ond color signal forming circuits operating on the 
basis of the first, second, and third color signals 
output from pixels corresponding to the first, second, 
and third color filters, and the color information for- 
ming means switching between the first and second 
color signal forming circuits in accordance with a 
frequency component in a scanning direction of the 
object image or a direction perpendicular to the 
scanning direction. 
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FIG. 6 
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